<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Mohanadarshan Vivekanandalingam on Medium]]></title>
        <description><![CDATA[Stories by Mohanadarshan Vivekanandalingam on Medium]]></description>
        <link>https://medium.com/@mohan-vive?source=rss-eba6b4500bcc------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*yXm_aPDcwnAuTLxubU7kug.jpeg</url>
            <title>Stories by Mohanadarshan Vivekanandalingam on Medium</title>
            <link>https://medium.com/@mohan-vive?source=rss-eba6b4500bcc------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 20:43:26 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@mohan-vive/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Thank You, Sanjiva — From the Heart]]></title>
            <link>https://mohan-vive.medium.com/thank-you-sanjiva-from-the-heart-49622bf247ee?source=rss-eba6b4500bcc------2</link>
            <guid isPermaLink="false">https://medium.com/p/49622bf247ee</guid>
            <category><![CDATA[wso2]]></category>
            <category><![CDATA[sanjiva]]></category>
            <dc:creator><![CDATA[Mohanadarshan Vivekanandalingam]]></dc:creator>
            <pubDate>Sat, 30 May 2026 01:51:24 GMT</pubDate>
            <atom:updated>2026-05-30T01:51:24.709Z</atom:updated>
            <content:encoded><![CDATA[<p>Sanjiva, Co-Founder and CEO of WSO2, is retiring from WSO2 at the end of May. It is an emotional moment for many of us, including me, who have worked at WSO2 for more than 13 years. Though I am not a frequent writer, I felt I should share a few thoughts about my journey at WSO2 and how much Sanjiva has impacted our lives.</p><p>I first came to know about Sanjiva in 2010, when I was in my second year at university and discovered the world of open source through his work online. In 2011, I sent him a LinkedIn invitation, not even sure if he would accept it. His work and vision inspired me to apply to WSO2. I still remember applying after every semester in my final year, and finally getting called for an interview after graduation. I was so excited to meet him during my final interview at the Flower Road office, in that corner room. I still have the offer email I received from him.</p><p>After joining WSO2, I had the chance to work closely with Sanjiva at different stages of my career. My first development task in the CEP team was assigned by him, writing a FIX transport for CEP. I still remember the code review by Sanjiva, Srinath, and Suho. As a new engineer, having the CEO review your code was quite nerve-wracking. But the empathy, patience, and encouragement he showed left a lasting impression on me. Even now, I feel I should have done a better job on that code 😀.</p><p>Being part of a small team like CEP gave us many opportunities to learn directly from him. The leadership, humility, and technical excellence he consistently demonstrated are qualities I will always remember. I learned a great deal from him, and many of those lessons still shape who I am today and how I interact with others.</p><p>I also fondly remember working on Ballerina, where I had the opportunity to work closely with him again to build streaming support for the language. Under his guidance, we took on an ambitious challenge and worked tirelessly to deliver it within a very tight timeline. There were only two of us on the team, learning something new every day and pushing forward because he believed in us and trusted us with the responsibility. I still remember the long hours of coding over those three months, often working seven days a week to make it happen. It was an intense journey, but one filled with learning, growth, and memories I will always cherish.</p><p>Have you ever challenged your leaders? At WSO2, we did that quite often. There were many occasions when I disagreed with Sanjiva and found myself in heated discussions with him. But those debates always stayed professional and never went beyond the room. I still remember discussions about reducing the size of the streaming team and eventually dispersing it. Even when we had different views, he always took the time to listen and consider every perspective. That openness to being challenged says a lot about his leadership.</p><p>I have always admired and learned from Sanjiva, the way he led the company, and how much he valued and trusted people and their ideas. Many of us at WSO2, including myself, stayed for over a decade because of the strong culture he helped build and nurture over the years.</p><p><a href="https://medium.com/u/3d4d7fd8d62d">Sanjiva Weerawarana</a>, THANK YOU SO MUCH. You have made our lives better. You will be deeply missed on this journey and will always be remembered in our success. I hope we continue to hear about your work and get another chance to connect with you again. Please Take care.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ghwT8LcwJSrZjsgq-htQuw.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cRkuxpV4GLGkxD8PSGv-Gg.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZF52LYGvvUZeBgSnv1Xvsw.jpeg" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=49622bf247ee" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Language Integrated Queries in Ballerina]]></title>
            <link>https://mohan-vive.medium.com/language-integrated-queries-in-ballerina-736eceacf968?source=rss-eba6b4500bcc------2</link>
            <guid isPermaLink="false">https://medium.com/p/736eceacf968</guid>
            <category><![CDATA[streaming]]></category>
            <category><![CDATA[sql-queries]]></category>
            <category><![CDATA[ballerina]]></category>
            <category><![CDATA[events]]></category>
            <dc:creator><![CDATA[Mohanadarshan Vivekanandalingam]]></dc:creator>
            <pubDate>Mon, 05 Oct 2020 10:51:35 GMT</pubDate>
            <atom:updated>2020-10-06T09:38:32.206Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*t4QQuBmmfKc4R8-IINf4Lg.jpeg" /><figcaption>Source: <a href="https://www.flickr.com/photos/coloradochris/22444880389/">https://www.flickr.com/photos/coloradochris/22444880389/</a></figcaption></figure><h3>Introduction</h3><p>Data processing is one of the critical components of enterprise integration. Even though there are various approaches to process data, query expressions are usually considered the go-to approach. Language integrated queries specify the logic in SQL-like syntax to process the data/events, and they are easy to write and understand due to the simplicity of the syntax. The <a href="https://ballerina.io/">Ballerina</a> programming language provides first-class support to write queries for data processing. In this article, we will discuss the query language support in Ballerina with some examples.</p><p>As of now, language integrated queries are supported for iterator implementations such as an array, map, stream, and table. There are two kinds of integrated queries that can be written in Ballerina — query expression, which allows generating a list, table, string or XML; stream and query action, which executes a set of statements for each element of the iterator.</p><h3>Query Expressions</h3><p>Query expressions contain a set of clauses similar to SQL to process the data. Query expressions must start with the `from` clause and can perform various operations such as filter, join, sort, limit, and projection. There are various SQL like clauses to perform these operations.</p><p><strong>Query Clauses</strong></p><ul><li><em>`from`</em> clause - Defines an input source/iterator</li><li>`<em>where</em>` clause - Defines a filtering condition</li><li>`<em>let</em>` clause - Defines a variable that could be used in the subsequent clauses</li><li>`<em>join</em>` clause - Defines an input source/iterator for joining purposes. It performs an inner or left outer equijoin.</li><li>`<em>order by</em>` clause - Defines the sorting logic</li><li>`<em>limit</em>` clause - Limits the number of elements passed to the next clause</li><li>`<em>select</em>` clause - Performs the projection. Outputs a list, table, string, XML, or stream.</li><li>`<em>on conflict</em>` clause - Allows only for a query expression that constructs a table with a key sequence to avoid duplicates</li></ul><p><strong><em>Example 1 - Query expression that constructs a list</em></strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ae34382795d748fae5d7fdf7908a5f3f/href">https://medium.com/media/ae34382795d748fae5d7fdf7908a5f3f/href</a></iframe><p><strong><em>Example 2 - Query expression that constructs a table</em></strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2845d595c728d0e03b6137a4c7f64ac6/href">https://medium.com/media/2845d595c728d0e03b6137a4c7f64ac6/href</a></iframe><p>Please refer to the Queries section in Ballerina By Examples for more information: <a href="https://ballerina.io/swan-lake/learn/by-example/">https://ballerina.io/swan-lake/learn/by-example/</a>.</p><h3>Query Actions</h3><p>Query actions are executed in the same way as the clauses in the query expression, but it doesn’t generate an output such as a list. It rather executes a set of statements defined by the user.</p><p><strong><em>Example 3 - Query action</em></strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3930a959ba3db006cc1aa075ccaee45d/href">https://medium.com/media/3930a959ba3db006cc1aa075ccaee45d/href</a></iframe><h3>Writing Integrated Queries</h3><p>The example below, related to an online product order use case, gives you a better understanding of Ballerina integrated queries. Refer the below code snippets for better understanding.</p><ul><li>There is an HTTP service to which users can request the product details. The payload is a JSON array that represents the product details.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2bAxvFFzoFz3zh-7e7JFUQ.png" /></figure><ul><li>Then the product details array (iterator) is manipulated using query expressions to find the price of each product in the iterator. Here, as per the query, we only consider the grocery products. In this process there is a `<em>join</em>` with the table which contains the product prices.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*guYvgK9luVOjeThj89DFrA.png" /></figure><ul><li>Next, an integrated query is used to calculate the total bill amount.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wtw_B_4kNHcFIu8LPgR1GA.png" /></figure><ul><li>Finally, the total bill amount is sent as the response to the HTTP request made.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dfUnUH_9RrfzaP-Jbdu_pA.png" /></figure><p>The complete Ballerina code can be found below.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/cae8aa0ebbbf468971264020ddcc5a75/href">https://medium.com/media/cae8aa0ebbbf468971264020ddcc5a75/href</a></iframe><h3>Try it Out</h3><p>Follow the steps below to try out the Ballerina example.</p><p>Note: These steps are given assuming the Ballerina runtime is already installed locally.</p><p>1. Copy the Ballerina code above and create a .bal file. (e.g., order_service.bal)</p><p>2. Then, execute the command below to start the HTTP service, which accepts product order requests.</p><pre>ballerina run order_service.bal</pre><p>3. Invoke the HTTP service by sending the product order details in the payload.</p><pre>curl -v http://localhost:9090/emart/orderProduct -d &#39;[{ &quot;Name&quot;: &quot;Flour&quot;, &quot;Id&quot;: 2345, &quot;Quantity&quot;: 2, &quot;ShoppingCardId&quot;: &quot;AXYN34523&quot;, &quot;Category&quot;: &quot;Grocery&quot;}, { &quot;Name&quot;: &quot;Carrot&quot;, &quot;Id&quot;: 3234, &quot;Quantity&quot;: 1, &quot;ShoppingCardId&quot;: &quot;AXYN34523&quot;, &quot;Category&quot;: &quot;Vegetable&quot;}]&#39; -H &quot;Content-Type:application/json&quot;</pre><p>4. Then, you will receive a response that gives the total grocery bill amount.</p><h3>Summary</h3><p>In this article, we have discussed Ballerina integrated query capabilities and provided a few examples for better understanding. As of now, integrated query support is available only for iterator objects and we are working on providing query support for unbounded streams (i.e., event stream processing).</p><p>For more information on Ballerina integrated queries, please refer to the Ballerina by Examples in <a href="https://ballerina.io/swan-lake/learn/by-example/">https://ballerina.io/swan-lake/learn/by-example/</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=736eceacf968" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Long Story Short: Siddhi - A view of a contributor]]></title>
            <link>https://mohan-vive.medium.com/long-story-short-siddhi-a-view-of-a-contributor-91aa747f3546?source=rss-eba6b4500bcc------2</link>
            <guid isPermaLink="false">https://medium.com/p/91aa747f3546</guid>
            <category><![CDATA[wso2]]></category>
            <category><![CDATA[stream-processing]]></category>
            <category><![CDATA[cloud-native]]></category>
            <category><![CDATA[complex-event-processing]]></category>
            <category><![CDATA[streaming]]></category>
            <dc:creator><![CDATA[Mohanadarshan Vivekanandalingam]]></dc:creator>
            <pubDate>Wed, 01 Jan 2020 14:12:33 GMT</pubDate>
            <atom:updated>2023-01-30T05:33:21.397Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gwE8IejJG2lQtZO8PSd4pw.jpeg" /><figcaption>Photo by VanveenJF on Unsplash</figcaption></figure><p>I have just completed 7 years at WSO2 on 1st January 2020. It was a great journey with a lot of memories to share. All these years, I have worked directly or indirectly related to Siddhi, been a contributor to Siddhi and Siddhi became part of my day to day life. I wanted to write up something around it and thought this would be a good time to share some words about Siddhi and its journey.</p><p>Siddhi in the Sinhala language means incidents (things that happen) and in Tamil language, it means knowledge. For whom, not aware of <a href="https://siddhi.io/">Siddhi</a>, it is a complex event processing and stream processing engine. Siddhi was started as a <a href="http://siddhi.sourceforge.net/">final year project</a> of the CSE department at the University of Moratuwa (UOM)with the support of WSO2 in 2011.</p><p>As a university <a href="https://dl.acm.org/doi/10.1145/2110486.2110493">final year project</a>, Siddhi was a success. Due to the prominence of the project, it became one of the main projects at WSO2 in 2012. Siddhi was used as one of the core engines with Esper and Drools in the WSO2 Complex Event Processor in 2012. I joined WSO2 on 1st January 2013 to Complex Event Processor Team (which develops and maintains Siddhi) with Suho (Suho was the one and only team member of the CEP team by that time). I still remember Sanjiva asked me to write a Fix adapter for Siddhi as my first task. I have joined WSO2 straight after my graduation; I never knew anything about CEP or Siddhi before joining the WSO2. I only got to know about Siddhi on my 2nd day at WSO2 (There was no preference option given to new joiners to select the team at that time). We have gone through ups and downs in this 7–8 years journey with Siddhi.</p><p>Until 2019, Siddhi was a supporting project/product to many other WSO2 products. It is used as a core component within many WSO2 products to cater to the use cases such as real-time analytics, alert generation, event-driven integration, throttling, etc.. In the following sections, I would like to highlight key milestones of Siddhi project and how it evolved.</p><p>If you checked the below figure, you will understand the WSO2 Analytics story over the years,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BoR_qWeiJLkA_-aRN4pGGg.png" /><figcaption>History of WSO2 Analytics Products</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dXGNG2cq0Iy9uVT3k_DvRA.png" /><figcaption>Architecture of Siddhi 1.x.x</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2Y0KKwIOWjBhvQk5zJQcTA.png" /><figcaption>Architecture of Siddhi 5.x.x</figcaption></figure><p>In the year 2013 to 2018, Siddhi capabilities are shipped through WSO2 Complex Event Processor, WSO2 Data Analytics Server, and WSO2 Stream Processor to serve the real-time analytics and streaming data integration requirements. In this period, the Siddhi engine has improved with better design, architecture, and implementation to provide high performance. Siddhi 1.x is released in 2012 and now, the Siddhi release version is 5.x.x. If you check the architecture diagrams of Siddhi 1.x and 5.x then you could understand the changes and improvements.</p><p>Over the period, we have revisited ourselves, taken feedback from the users and inspired by other vendors in the market to improve the Siddhi engine.</p><h4>Role of Siddhi at WSO2</h4><ul><li>Siddhi powered and powers the <a href="https://wso2.com/analytics-and-stream-processing/">WSO2 Analytics products</a> such as WSO2 Complex Event Processor, WSO2 Data Analytics Server, and WSO2 Stream Processor.</li><li>Streaming data integration requirements of <a href="https://wso2.com/integration/streaming-integrator/">WSO2 Enterprise Integrator</a> is powered by the Siddhi engine.</li><li>WSO2 APIM, EI and IS analytics runtimes are powered by the Siddhi engine.</li><li><a href="https://wso2.com/library/articles/2016/09/article-introducing-wso2-api-managers-throttling-implementation-architecture/">WSO2 API Manager throttling requirements</a> (APIM Traffic Manager) are built on top of the Siddhi engine.</li><li><a href="https://is.docs.wso2.com/en/5.9.0/learn/adaptive-authentication/">Adaptive authentication</a> capabilities of WSO2 Identity Server is powered by Siddhi engine.</li></ul><h4>Who adopted Siddhi</h4><p>There is a lot of adoption happened around Siddhi even though Siddhi was not positioned as a first-class product. See the below figure to find the users of Siddhi.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Bs-pKEtxAZpyS68YUpFcbQ.png" /><figcaption>Users of Siddhi</figcaption></figure><ul><li>Siddhi used as the core component of Uber’s fraud detection system.</li></ul><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FnncxYGD6m7E%3Ffeature%3Doembed&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DnncxYGD6m7E&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FnncxYGD6m7E%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/e2fdc9810afa0e11d5fbf0e427cf5f67/href">https://medium.com/media/e2fdc9810afa0e11d5fbf0e427cf5f67/href</a></iframe><ul><li>Siddhi is part of <a href="https://eagle.apache.org/docs/">Apache Eagle</a>, a distributed real-time monitoring and alerting engine. This was initiated by eBay and Paypal and currently used in house.</li></ul><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FBueiob7S6d4%3Ffeature%3Doembed&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DBueiob7S6d4&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FBueiob7S6d4%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/2d908a7151d42145496327de42612a18/href">https://medium.com/media/2d908a7151d42145496327de42612a18/href</a></iframe><ul><li>Siddhi is a part of the CSI Piemonte’s <a href="https://github.com/csipiemonte/yucca-realtime">YUCCA smart data platform</a>.</li><li>Siddhi is a part of the <a href="https://doc.punchplatform.com/4.0.2/src/doc/03_USER_GUIDE/04_Punch_Programming/03_Punch_Siddhi.html#overview">Punch Platform</a>.</li></ul><p>Other than above, it has been used by many <a href="https://scholar.google.com/scholar?hl=en&amp;as_sdt=0%2C5&amp;q=Siddhi%2BCEP&amp;btnG=">undergraduate, postgraduate and doctorate studies</a> as a complex event processing engine as well.</p><h4>Where are we heading now (From 2019 onwards)?</h4><p>In early 2019, it was decided to position Siddhi as an independent OSS project at WSO2. As a step, five engineers (<a href="https://medium.com/u/256afc94c5d3">Suhothayan Sriskandarajah</a>, <a href="https://medium.com/u/37b50b68866e">Chiran Fernando</a>, <a href="https://medium.com/u/6a9fa9c7a519">Niveathika Rajendran</a>, <a href="https://medium.com/u/ff95aebdaab5">Buddhi Kothalawala</a> and myself) started working on the Siddhi OSS project. In the process, Siddhi is moved to separate GitHub organization as <a href="https://github.com/siddhi-io">siddhi-io</a> for better adoption and management and also Siddhi team built a <a href="http://siddhi.io/">Siddhi GitHub site</a> to spread the news about Siddhi. The primary goal of the siddhi-io is to build a Cloud Native Stream Processor to overcome the complication of building and running streaming applications in Kubernetes and etc…</p><p>You could refer to my previous article to get some understanding on Cloud Native Stream Processing and what Siddhi Cloud Native Stream Processor does,</p><p><a href="https://medium.com/@mohan.vive/why-do-you-need-cloud-native-stream-processing-648c5b86ed98">Why do you need Cloud Native Stream Processing ?</a></p><p>The siddhi-io project consists of nearly <a href="https://siddhi.io/community/committers/">30 committers</a> but the core project team who is working on the project in full time is small in size. Since it is a small team all the engineers in the team get a chance to work on different areas of the project; this includes project development, testing, automation, documentation, evangelism efforts, and community support.</p><p>Importantly, we always engage with community users and contributors through <a href="https://groups.google.com/forum/#!forum/siddhi-dev">siddhi-dev google mail group</a> and <a href="https://siddhi.io/community/#community-collaboration">Siddhi Slack channel</a> to provide better help and promote community collaboration towards the project. For example, the idea for the Siddhi project logo is decided by the votes of the community members through the siddhi-dev mail thread.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/682/1*Ra69qPJlKTyz6t2L89hnog.png" /><figcaption>Siddhi-io Logo Proposals</figcaption></figure><p>Check out the <a href="https://siddhi.io/en/v5.1/development/roadmap/">road map</a> of Siddhi project and provide your suggestions through siddhi-dev google mail group or Siddhi Slack channel.</p><h4>Key Milestones of Siddhi</h4><ul><li>In 2012, Siddhi 1.x won the bronze award in the Tertiary category at the National Best Quality Software Awards 2011.</li><li>Siddhi is the only generic complex event processing engine which able to become a finalist in the Distributed Event-based Systems (<a href="https://debs.org/">DEBS</a>) conference grand challenge from 2014 to 2017 consequently (years which Siddhi has participated).</li><li>In 2015, Siddhi has utilized by <a href="https://www.youtube.com/watch?v=i4QxJIHrfOY">Uber</a> to build the scalable event processing solution for fraud detection.</li><li>In 2015, <a href="https://wso2.com/blogs/thesource/2015/10/wso2con-insights-experian-uses-wso2-to-uncover-credit-intelligence/">Experian</a> utilized Siddhi CEP engine to uncover credit intelligence use cases.</li><li>Siddhi Won the <a href="https://blog.tfl.gov.uk/2016/10/04/data-in-motion-hack-week-event-review/">TFL’s data in motion hackathon</a> in 2016 to find solutions to some of London’s key transport challenges through the innovation and creativity of developers working with our open data.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SvdRUbACG2N7APzIvXAvcA.jpeg" /></figure><ul><li>In February 2019, Siddhi <a href="https://medium.com/siddhi-io">medium tech blog</a> is introduced to provide official blogs and articles of siddhi-io.</li><li>In June 2019, <a href="https://github.com/haoch">Hao Chen</a> became the first external committer to siddhi-io project based on his contribution towards Siddhi and integration of Siddhi to Apache Bahir and Apache Eagle.</li><li>In July 2019, Siddhi listed in the CNCF landscape as the streaming and messaging technology.</li></ul><p><a href="https://landscape.cncf.io/selected=siddhi">CNCF Cloud Native Interactive Landscape</a></p><ul><li>In July 2019, Siddhi 5.1.x <a href="https://siddhi.io/en/v5.1/release-notes/">release</a> was done with a lot of refactoring for better performance and architecture to support building cloud native streaming applications.</li><li>In August 2019, <a href="https://siddhi.io/community/#community-collaboration">Siddhi Slack channel</a> is initiated to provide better support for community users and contributors.</li><li>In September 2019, Siddhi community call is introduced and it is scheduled to run weekly on Tuesday at 7.30 AM PST.</li><li>In December 2019, Siddhi earned the <a href="https://bestpractices.coreinfrastructure.org/en/projects/2974">CII Best Practices Badge</a> from Linux Foundation Core Infrastructure Initiative.</li><li>In 2019, Siddhi GitHub repo got nearly 400 new Stargazers (which is the highest for Siddhi) due to organic traffic and content based evangelism effort.</li></ul><h4>Request for OSS Users</h4><p>It is always challenging to keep an open source project active. Every user must understand the ethics of an OSS user. As an OSS user, it is important that he/she contribute back to the project. While it’s easy to just use without giving back but it is your responsibility to ensure that these projects continue to improve and stay healthy. This does not mean that you should definitely pay something as such. You could contribute back to the project in the following ways,</p><ul><li>Give a star to the GitHub<a href="https://github.com/siddhi-io/siddhi"> repository</a> if you felt that the project is beneficial for you.</li><li>Send a PR if you made an improvement for documentation/code or developed a feature or fixed a bug.</li><li>Raise a GitHub issue if you found any bug while you are using the project/product.</li><li>Respond to the queries raised by other community users/contributors.</li><li>Spread the news about the OSS project (Siddhi).</li><li>Share your use case as a reference (or provide a testimonial).</li></ul><blockquote>In the real open source, you have to give back as equal to what you take.</blockquote><p>Thanks for reading the article.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=91aa747f3546" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why do you need Cloud Native Stream Processing ?]]></title>
            <link>https://mohan-vive.medium.com/why-do-you-need-cloud-native-stream-processing-648c5b86ed98?source=rss-eba6b4500bcc------2</link>
            <guid isPermaLink="false">https://medium.com/p/648c5b86ed98</guid>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[stream-processing]]></category>
            <category><![CDATA[complex-event-processing]]></category>
            <category><![CDATA[cloud-native]]></category>
            <dc:creator><![CDATA[Mohanadarshan Vivekanandalingam]]></dc:creator>
            <pubDate>Mon, 30 Dec 2019 10:12:24 GMT</pubDate>
            <atom:updated>2019-12-31T12:40:16.221Z</atom:updated>
            <content:encoded><![CDATA[<h3>Why Do You Need Cloud Native Stream Processing ?</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KQaqsm2w88LtTnOcYAQjlg.jpeg" /><figcaption>Photo by Manuel M. Almeida on Unsplash</figcaption></figure><p>This article is all about Stream Processing and Cloud Nativeness. You might already have an idea about this article based on the title. Then, without much sugar coating let’s discuss the subject.</p><p><strong>Cloud Nativeness</strong> one of the prominent term or buzz words you could hear in the tech industry in recent days. There are a lot of organizations are revisiting their enterprise architecture and system to move them to the cloud. For most of the organization, moving into the Cloud is considered as the next milestone in the digital transformation effort.</p><p>OK, now. What is meant by Cloud Nativeness?</p><blockquote>Cloud-native is an approach to building and running applications that exploits the advantages of the cloud computing delivery model.</blockquote><p>There are a lot of things are happening around Cloud Native applications, infrastructure, and ecosystem with the support and moderation of <a href="https://www.cncf.io/">CNCF</a>.</p><p>As per <a href="https://blogs.gartner.com/tony-iams/containers-serverless-computing-pave-way-cloud%20native-infrastructure/">Gartner</a>, Cloud native infrastructure is growing rapidly,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/812/1*Be9QDUv57CCkiLcHPBOM9g.jpeg" /></figure><p>Also <a href="https://go.forrester.com/blogs/predictions-2020-cloud/">checkout</a> the promising predictions made by Forrester regarding cloud computing and infrastructure for 2020.</p><p>Currently, you can find a lot of legacy services or server-based services (monolithic services) that are moving to microservices as a step to move into the cloud. A lot of organizations are succeeded in that as well. In the high level below are the advantages that you can gain when moving into the cloud.</p><p><strong>Cost effectiveness</strong> - Let’s consider the on-premise server environments, there are situations that servers are Idle (or not fully utilized) because traffic varies based on the time, etc.. Then, it ended up spending up resources and money unnecessary without any use. With the help of cloud native applications (and its container architecture), unused resources are eliminated.</p><p><strong>Reliability</strong> - Due to the container nature of the cloud native application, they are self-sufficient and works uninterrupted. Also, since the application is built as a separate microservice for each independent business function, failure in one container will not affect other business functions.</p><p><strong>Scalability</strong> - Each business function is built into independent micro services and they are loosely coupled. Due to that, each of those microservices can be scaled independently as separate containers. Cloud platforms such as Kubernetes, open shift, etc also support to achieve this natively without much hassle.</p><p>Now, you can understand that cloud native application provides some unique advantages over the on-premise applications/services. It is straightforward and easy to move traditional monolithic stateless applications to the cloud because cloud infrastructure highly supports or fits into that. Also, as you aware Kubernetes plays an important role in achieving cloud nativeness and various technologies are gathering around Kubernetes to build cloud native applications</p><h3>What about Stream Processing?</h3><p>Stream processing applications are part of the enterprise system which is responsible for event-driven integrations and real-time analytics, then with this cloud native evolution, those stream processing applications also need to be moved into the cloud to gain the advantages of the cloud native transformation; it is not beneficial to run stateless services/applications in cloud and stream processing applications in on-premise. But moving stream processing applications which are stateful into the cloud is not easy as stateless services. To play in the cloud native space, a stream processor must-have characteristics such as being lightweight, loosely coupled and adhering to the agile dev-ops process. However, most of the traditional stream processors are heavy and depend on bulky monolithic technologies which makes them harder moving to the cloud. Below are some of the areas that blocking existing Stream Processors to move in to cloud.</p><ul><li>Majority of the existing Stream Processors are heavy &amp; built on top of bulky monolithic technologies</li><li>Node to node traffic is high</li><li>Based on master-slave architecture</li><li>Streaming applications are stateful</li><li>Cannot scale the nodes independently</li></ul><p>As given above, some design/architectural factors of the existing Stream Processors are not cloud-friendly and it hinders to move into the cloud. Then, Stream Processors should have an architecture that supports cloud deployment. But the stateful nature of the streaming applications is unavoidable in stream processing context then it is important to invest time/energy to achieve those requirements in cloud frameworks such as Kubernetes.</p><h3>Why Siddhi?</h3><p><a href="https://siddhi.io/">Siddhi</a> Stream Processor is designed to build stream processing applications and run them in the cloud. It is built on top Siddhi stream processing library which is lightweight, it can boot up within a few seconds and able to process more than 100K events per second. Due to the micro nature of the Siddhi architecture, it allows you to deploy as containers. Siddhi provides native support to deploy a streaming application in Kubernetes with Siddhi K8s operator. Siddhi Kubernetes deployment patterns are built on top of features provided by Kubernetes and other cloud native technologies such as NATS and Prometheus.</p><p>Siddhi deployment patterns are not based on master-slave architecture and there is no traffic involved between the nodes thus container-based deployment becomes easy and it leads to deploying Siddhi in Kubernetes natively. In Kubernetes, Siddhi supports to run stateful stream processing applications by involving a messaging system such as NATS and state snapshot persistence with a volume mount.</p><p>Let’s discuss more Siddhi Cloud Native Stream Processor and how it achieves scalability, reliability and other benefits of cloud infrastructure in the next article. You can refer to the Siddhi documentation <a href="https://siddhi.io/en/v5.1/docs/siddhi-as-a-kubernetes-microservice/">here</a> to get more understand on this.</p><p>Thank you..</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=648c5b86ed98" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What if your Political Beliefs becomes your Religion]]></title>
            <link>https://mohan-vive.medium.com/what-if-your-political-beliefs-becomes-your-religion-ee4be3aa2e9d?source=rss-eba6b4500bcc------2</link>
            <guid isPermaLink="false">https://medium.com/p/ee4be3aa2e9d</guid>
            <category><![CDATA[elections]]></category>
            <category><![CDATA[religion]]></category>
            <category><![CDATA[belief]]></category>
            <category><![CDATA[politics]]></category>
            <dc:creator><![CDATA[Mohanadarshan Vivekanandalingam]]></dc:creator>
            <pubDate>Fri, 15 Nov 2019 19:20:44 GMT</pubDate>
            <atom:updated>2021-05-10T15:48:56.910Z</atom:updated>
            <content:encoded><![CDATA[<h3>What if your Political Beliefs becomes your Religion?</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/848/1*ZWdqNK6bY1WiDy4I4zdymg.jpeg" /><figcaption><a href="https://www.flickr.com/photos/jayqueue/163798580/">https://www.flickr.com/photos/jayqueue/163798580/</a></figcaption></figure><p>OK, we are so close to the Sri Lankan presidential election 2019, just a few hours to the election day.</p><p>This blog post is not written to support or attack one specific presidential candidate. That does not mean that I am a so-called “neutral” person; I also have my political view, which many accept and reject. This medium post shares my observations and my opinion regarding social media usage and political propaganda around it.</p><h3>Supporter vs Neutral vs Devotee</h3><p>We can divide the voters into three segments; they are supporters, neutral persons, and devotees. A supporter is always praising the good things done by the group/party that he/she supports and also attacks the opposition when they do any bad thing. This person will not express any view when there is something wrong done by the group/party that he/she supports. At the same time, this person will be silent when the opposition does a good.</p><p>A neutral person is always praising if any party or group does anything good. At the same time, this person also attacks if any group/party does anything bad. But, IMO there is no someone called a neutral person. Any voter has their own political view and belief. The only thing is this person will not engage in discussions in public forums by supporting or attacking a party/group.</p><p>Next, the most important and interesting category; “Devotee”. People belong to this category are always defend the group/party which he believes. A devotee will not criticize the bad thing that their master (or party) did; they always defend and take examples from the opposition to defend it.</p><p>All of you came across different kinds of people who belong to one of the above three categories (you can do a self-analysis). Being a supporter or being a so-called “neutral” person is acceptable in a civilized community, but devotees are so harmful to society. I have come across many posts, and hatred spread towards the people who have different opinions, and it is unbelievable. This clearly shows that his/her political beliefs become their religion. They are not ready to question, argue or listen to different opinions because they get hurt when they hear something bad/allegation about their master/god.</p><p>I am a firm believer that any religious view should be questioned without blindly following it. You will only understand the truth and enjoy it when you understand things to the core. This is the same for political devotees as well. It is well explained in the below figure too. Found it on Facebook btw.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/652/1*KU8vFSlxTRC537tLUx5VCw.jpeg" /></figure><h3>Understanding the People’s need</h3><p>You can divide the voters from another perspective as well; they are traditional voters and new-age voters. Traditional voters are the people that we call “kepuwath kola” or “kepuwath nil”. We cannot change this category and don’t spend time changing them. You will not even have a 1% of positive chance for that.</p><p>New-age voters are the people who influence and deciding factor in any election. This category of voters has various needs and expectations. Simply, a person who is not sure about the next meal looks for some basic needs such as food, work, etc. But, a person who has covered his/her basic needs and looks for esteem or self-actualization needs. People’s needs differ from person to person. If you like to know more about this, then please have a quick look at my previous medium post on “<a href="https://medium.com/@mohan.vive/learning-outcome-of-maslows-hierarchy-of-needs-a6ff0cd3baa9">Maslows hierarchy of needs</a>”.</p><p>These needs play an essential role in your political beliefs. You cannot expect your friend or buddy to have the same political beliefs because their needs are totally different from yours. A person who consumes food for its taste will not understand the needs of a person who eats due to hunger.</p><h3>We don’t need that 225</h3><p>This is one of the very common slogans you have seen in social media in the recent past. I would say this could be the most stupid and illiterate comment that one could say. Let’s ignore that 29, which are appointed through the national list but just consider that 196 members are selected by the supreme power called “Public”. These 196 MPs are a sample of the general public. You cannot change the selections without changing the whole set of values. We cannot change those 255 MPs unless the general public changes their attitude and beliefs. This is not something impossible. Let’s take an example from our neighbouring country India, the AAM AAdmi party, which was formed by a group of social activists able to capture the power of New Delhi state within six months after forming the party defeating two main strong opposition parties.</p><p><a href="https://www.bbc.com/news/world-asia-india-31294500">India anti-corruption party routs BJP</a></p><p>But, I believe that it will take a considerable amount of time for us to expect such a change; it would be another few decades.</p><h3>My Perspective</h3><p>Even though my mother language is Tamil, I was fortunate to learn and exercise Sinhala. This is mainly because I had the social pressure to learn, and also I have been surrounded by a lot of friends who speak Sinhala (But, I always proud that I can speak proper Sinhala than them :D ). I usually get to see Facebook ads, interviews, etc.; seriously, the number of contradictions that I could find in them is ridiculous. I witnessed a Facebook ad published by one of the leading candidates of the presidential election (in his Tamil Facebook page) contracts with his stand, which is well known in the south. This implies candidates are moving towards gaining more and more votes rather than focusing on their political stand.<br> <br>I am used this election period to learn and understand the state of the social media community, and it is outrageous. I also witnessed many of them use very sensitive matters as a trump card to defend their political beliefs. I felt a bit worried and laughed at some posts because that shown the state of the inner soul irrespective of your position or qualifications; you have made your family &amp; parents proud, it clearly indicates your upbringing 😉. Take a few steps back, do a self-analysis, and then you will understand how much hatred you have spread to your fellow community and friends. This also reiterates that Sri Lanka cannot see a prime minister or president from a minority community in the near future.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QKHVHmL_vmZs6Bf0LoklsA.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@tchickmcclure?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">T. Chick McClure</a> on <a href="https://unsplash.com/s/photos/politics-vs-religion?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><p>In another perspective, I firmly believe each and every person has their likes and dislikes, but what I could understand is <strong>most of the people are not keen on who should win rather they are so keen on who should not win </strong>😀. I am pretty sure this will be reflected in the coming presidential election.</p><p>Finally, this medium post is not written to influence any of you, but rather I felt that I have deserted in this <strong>hatred political mudslinging</strong> and wanted to register my view. Thanks.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ee4be3aa2e9d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Learning outcome of Maslow’s Hierarchy of Needs]]></title>
            <link>https://mohan-vive.medium.com/learning-outcome-of-maslows-hierarchy-of-needs-a6ff0cd3baa9?source=rss-eba6b4500bcc------2</link>
            <guid isPermaLink="false">https://medium.com/p/a6ff0cd3baa9</guid>
            <category><![CDATA[psychology]]></category>
            <category><![CDATA[maslows-hierarchy]]></category>
            <category><![CDATA[need]]></category>
            <category><![CDATA[voters]]></category>
            <category><![CDATA[self-actualization]]></category>
            <dc:creator><![CDATA[Mohanadarshan Vivekanandalingam]]></dc:creator>
            <pubDate>Sun, 28 Apr 2019 10:24:38 GMT</pubDate>
            <atom:updated>2019-04-28T10:51:07.903Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*e5ZZ73nPKqrE5_I3zVgR7Q.jpeg" /></figure><p>I am writing this article after few months but this was one of the topics that I was looking for a long time to share my views. Interestingly this article is not directly related to any technical content of paradigm.</p><p>Before moving to the details, I would like to highlight a few incidents that I came across over the past few months. Please find them below.</p><h4>1) Increase of vote selling in India (Bribe for vote)</h4><ul><li><a href="https://www.deccanherald.com/specials/sunday-spotlight/vote-buying-season-is-here-724785.html">https://www.deccanherald.com/specials/sunday-spotlight/vote-buying-season-is-here-724785.html</a></li><li><a href="https://www.independent.co.uk/news/world/asia/india-elections-bribes-voters-weddings-newspapers-cash-modi-bjp-congress-a8746266.html">https://www.independent.co.uk/news/world/asia/india-elections-bribes-voters-weddings-newspapers-cash-modi-bjp-congress-a8746266.html</a></li><li><a href="https://www.thehindubusinessline.com/blink/cover/politicians-bribe-with-gold-cash-liquor-and-freebies-this-general-election/article26809036.ece">https://www.thehindubusinessline.com/blink/cover/politicians-bribe-with-gold-cash-liquor-and-freebies-this-general-election/article26809036.ece</a></li><li><a href="https://www.premiumtimesng.com/foreign/world-foreign/322394-elections-india-seizes-cash-liquor-drugs-meant-to-bribe-voters.html">https://www.premiumtimesng.com/foreign/world-foreign/322394-elections-india-seizes-cash-liquor-drugs-meant-to-bribe-voters.html</a></li></ul><h4>2) Easter Sunday bombing in Sri Lanka, terrorists are identified as well educated and financially capable/well settled.</h4><ul><li><a href="https://www.tellerreport.com/news/--attacks-in-sri-lanka--two-assassins-were-apparently-sons-of-a-rich-entrepreneur-.r1Zr73JksV.html">https://www.tellerreport.com/news/--attacks-in-sri-lanka--two-assassins-were-apparently-sons-of-a-rich-entrepreneur-.r1Zr73JksV.html</a></li><li><a href="https://www.washingtonpost.com/world/asia_pacific/sri-lanka-reveals-identities-of-suicide-bombers-behind-easter-massacres/2019/04/24/5df35f60-6611-11e9-a698-2a8f808c9cfb_story.html?noredirect=on">https://www.washingtonpost.com/world/asia_pacific/sri-lanka-reveals-identities-of-suicide-bombers-behind-easter-massacres/2019/04/24/5df35f60-6611-11e9-a698-2a8f808c9cfb_story.html?noredirect=on</a></li><li><a href="https://www.npr.org/2019/04/24/716664302/bombers-who-carried-out-attacks-in-sri-lanka-were-well-educated-minister-says">https://www.npr.org/2019/04/24/716664302/bombers-who-carried-out-attacks-in-sri-lanka-were-well-educated-minister-says</a></li></ul><p>In the above, I have taken two different incidents. You might be wondering what is the relationship between them. As you thought there is no direct relationship between the incidents but at the end of this article you view might change at least a bit.</p><p>If you take the incident #1, it is one of the struggling problems in Indian election as of now. Election commission of India trying their best to stop bribing voters but obviously not succeeded enough. This is not the first time that we came across such an incident. Nearly, a year back this topic was discussed in publicly on R.K. Nagar by-election, Tamil Nadu; 20 rupees currency note used as a token for 8000–10000 INR for a voter. Check more details in the below references.</p><ul><li><a href="http://www.newindianexpress.com/states/tamil-nadu/2018/apr/30/women-wave-rs-20-notes-remind-ttv-dhinakaran-of-promised-money-during-rk-nagar-by-election-1808129.html">Women wave Rs 20 notes, remind TTV Dhinakaran of &#39;promised&#39; money during RK Nagar by-election</a></li><li><a href="https://www.indiatoday.in/india/story/battle-for-jayas-rk-nagar-bribing-voters-ttv-dinakaran-aiadmk-amma-968853-2017-03-31">Battle for Jaya&#39;s RK Nagar: Man arrested for bribing voters, TTV Dinakaran denies he&#39;s a supporter</a></li></ul><p>after this incident, there were a lot of discussions against bribing the voters/selling votes. There were various propaganda evolved saying voters should look at the ideology and facts of the candidate when casting their vote rather looking for money or freebies.</p><p>If you consider the incident #2, which is so pathetic have a different side of the story. As per the initial investigation, terrorists are identified as well educated and financially settled. They haven’t done these suicide bomb attacks for money rather they had a different motive and target.</p><p>Now, we can understand some people have basic needs as per incident #1 and some have totally different needs as per incident #2. This difference of need is clearly explained by <strong>Maslow’s hierarchy of needs</strong> and it can provide answers for many of our questions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8dG8fgA6VdM8kCI-Nl50pQ.jpeg" /></figure><h3>Discussion &amp; Take Away</h3><p>Now, Let’s take the incident #1, the majority of the people who have sold their votes are from lower and lower-middle class people who are financially struggling for their existence. Now, people who are doing propaganda and awareness against bribe for votes are typically from the upper and upper-middle-class community.</p><p>In simple words, people who are struggling for their next meal don’t look for self-actualization or esteem needs rather they have basic physiological and safety needs. Then they will not look at the ideology or facts of a candidate rather they look at what type of financial and social benefit they will get when casting a vote.</p><p>Let’s consider others who already achieved their physiological, safety, love/belonging needs (as like incident #2); their needs are totally different from a person who works hard for his/her next meal. When they achieved their basic needs, they’ll get so bored and there is a question arose as what is next? This is a very important stage, this kind of people looking for esteem and self-actualization needs. But, at this stage, they need clear guidance to achieve the esteem and self-actualization needs in a way that society also gets benefit out of it, if not they could go out of track as similar to incident #2.</p><p>Then, in summary, needs are different from person to person. If you are financially well settled then your needs are different from a person who is working hard for his/her next meal. Person who is working hard to achieve the basic needs will not listen to your ideology/facts.</p><p>Just my few cents…</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a6ff0cd3baa9" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Story Behind Ballerina Streams]]></title>
            <link>https://mohan-vive.medium.com/story-behind-ballerina-streams-826524c0a6c8?source=rss-eba6b4500bcc------2</link>
            <guid isPermaLink="false">https://medium.com/p/826524c0a6c8</guid>
            <category><![CDATA[integration]]></category>
            <category><![CDATA[ballerina]]></category>
            <category><![CDATA[wso2]]></category>
            <category><![CDATA[siddhi]]></category>
            <category><![CDATA[stream-processing]]></category>
            <dc:creator><![CDATA[Mohanadarshan Vivekanandalingam]]></dc:creator>
            <pubDate>Sun, 09 Dec 2018 04:08:54 GMT</pubDate>
            <atom:updated>2019-04-28T10:51:37.775Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bWsQ3cr961tMhgBT8axpmg.jpeg" /><figcaption><a href="https://www.flickr.com/photos/jurvetson/4634541472/">https://www.flickr.com/photos/jurvetson/4634541472/</a></figcaption></figure><p>In my previous <a href="https://medium.com/@mohan.vive/stream-processing-in-enterprise-integration-world-f15989938951?fbclid=IwAR1tu3yMSxlkb7mN-icatXJfuT4CL60orWLuPMk8pyEFSsAOLqCW2UwJPko">post</a>, I have discussed on Stream Processing and its need in the integration world; in this post, I am mainly focusing on Ballerina Streams and how it is evolved over the time, promising capabilities and its direction.</p><h3>The Start</h3><p>After the evolution of Ballerina programming language, we have identified the need for the Streaming capabilities in the language. If you have a question on “why” then better you read the <a href="https://medium.com/@mohan.vive/stream-processing-in-enterprise-integration-world-f15989938951?fbclid=IwAR1tu3yMSxlkb7mN-icatXJfuT4CL60orWLuPMk8pyEFSsAOLqCW2UwJPko">previous article</a>.</p><p>As the first step, we wanted to bring the streaming capabilities to Ballerina language within a very short time period. As some of you aware, we already had an in-house built and most powerful Complex Event Processing engine; which is called as <a href="https://github.com/wso2/siddhi">Siddhi</a>. After few rounds of heated discussions we came up with a streaming language syntax for Ballerina; it was more like a marriage between Siddhi query syntax and Ballerina programming syntax.</p><p><strong>Siddhi Streaming Query Syntax</strong></p><pre><strong>from</strong> TempStream#window.time(10 min)<br><strong>select</strong> avg(temp) as avgTemp, roomNo, deviceID<br><strong>insert</strong> <strong>into</strong> AvgTempStream;</pre><p><strong>Ballerina Streaming Query Syntax</strong></p><pre><strong>forever </strong>{<br>    <strong>from </strong>tempStream <strong>window </strong>time(600000)<br>    <strong>select </strong>avg(tempStream.temp) <strong>as </strong>avgTemp, tempStream.roomNo,<br>    tempStream.deviceID<br>    <strong>=&gt; </strong>(AvgTemperature [] values) {<br>        foreach var value <strong>in </strong>values {<br>            avgTempStream.publish(value);<br>        }<br>    }<br>}</pre><p>Within 2 months time period (with 2 engineers), we were able to integrate the Siddhi CEP engine with Ballerina language. This is more likely a marathon implementation. In this approach when a user writes the streaming query in Ballerina syntax it will be converted into the Siddhi query and pushed into the Siddhi runtime as shown in below figure.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/1*I_xdne_JS1aK1QuWbNk_hw.jpeg" /><figcaption>Query Flow in Siddhi Based Streaming Implementation</figcaption></figure><p>Even Though, this approach caters necessary capabilities that required for Streaming needs; we also identified some areas that we wanted to improve. Below are the key reasons behind it.</p><ul><li>Inability to access the runtime variables and functions which are in Ballerina runtime from Streaming queries which runs in Siddhi engine.</li><li>Underneath implementation language of Siddhi is not full comparable with Ballerina.</li></ul><p>Due to these reasons, our direction changed towards implementing the streaming processing capabilities from the scratch using the Ballerina language itself. This also helps us to prove the feature richness of the Ballerina language and stand as an example. <strong>Now, rest of this post discusses on Ballerina Streams implementation which done using Ballerina language itself.</strong></p><h3>Streaming Implementation — For Ballerina By Ballerina</h3><p>It was decided to implement the streaming capabilities for Ballerina programming language using the Ballerina itself. In this approach, we have implemented the streaming constructs such as windows, aggregators, groupby and etc.. as reusable models and they were used to build complete queries.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/956/1*uOFUMZnYmGYCgVCfMGGmDw.jpeg" /><figcaption>Query Flow in Native Ballerina Streaming Implementation</figcaption></figure><p>As shown in above Figure, streaming query written by a developer goes through few phases at the compile time. It passes through the lexical analyzer, parser, semantic analyzer, code analyzer, taint analyzer, desugar and code generator. Here, when streaming query reaches to the Desugar phase, the respective query is converted into typical Ballerina first-class constructs and code is converted into another form by removing the syntactic sugar. Then, the desugared code is code gen in the code generator phase. In the desugaring phase, the pre-built streaming models such as windows, aggregators and etc.. are used to perform this.</p><p>Note: If you like to understand more about the Ballerina Compiler and Ballerina Virtual Machine then please refer below posts written by<a href="https://medium.com/@sameerajayasoma"> Sameera Jayasoma</a>, lead architect of Ballerina Compiler and Runtime.</p><ul><li><a href="https://medium.com/@sameerajayasoma/ballerina-compiler-design-3406acc2476c">https://medium.com/@sameerajayasoma/ballerina-compiler-design-3406acc2476c</a></li><li><a href="https://medium.com/@sameerajayasoma/ballerina-runtime-evolution-f82305e4ab8e">https://medium.com/@sameerajayasoma/ballerina-runtime-evolution-f82305e4ab8e</a></li></ul><p>As per the above approach, the streaming query written by a user is converted into typical Ballerina code. For example,</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/cc7f470899a4abde7fd2c623870573c6/href">https://medium.com/media/cc7f470899a4abde7fd2c623870573c6/href</a></iframe><p>Above streaming query will be desugared into below Ballerina code and execution continues.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/9a7532b8fc32759addfd7d6cda3ccf52/href">https://medium.com/media/9a7532b8fc32759addfd7d6cda3ccf52/href</a></iframe><p>As you can see, a simple streaming query which is written by the developer is converted into a few lines of code. This reduces the complexity of dealing the huge code for stream processing and provides a very simple and easily understandable layer for the developers.</p><h3>Ballerina Streaming Capabilities</h3><p>Ballerina Streaming Implementation supports below streaming constructs. These constructs are required enough to achieve most of the streaming needs in the integration world.</p><h4>Projection</h4><p>This involves selecting only some of the attributes from the input stream to be inserted into an output stream, renaming the attributes, add constant values to the output stream and perform logical or mathematical operations.</p><p><em>Example Query</em></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d858291114b750de0c30f724d955eba7/href">https://medium.com/media/d858291114b750de0c30f724d955eba7/href</a></iframe><h4>Filter</h4><p>Filters are included in queries to filter information from input streams based on a specified condition.</p><p>Purpose : A filter allows you to separate events that match a specific condition as the output or for further processing.</p><p><em>Example Query</em></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e472b34ccf6f990aff8285caa9e4b1a1/href">https://medium.com/media/e472b34ccf6f990aff8285caa9e4b1a1/href</a></iframe><h4>Window</h4><p>Windows allow you to capture a subset of events based on a specific criterion from an input stream for calculation. Each input stream can only have a maximum of one window.</p><p>Purpose : To create subsets of events within a stream based on time duration, number of events, etc for processing. A window can operate in a sliding or tumbling (batch) manner.</p><p>At the moment, Ballerina Streaming supports below window implementations.</p><ul><li>time</li><li>timeBatch</li><li>timeLength</li><li>length</li><li>lengthBatch</li><li>externalTime</li><li>externalTimeBatch</li><li>uniqueLength</li><li>delayWindow</li><li>sort</li></ul><p><em>Example Query</em></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/08cbfb1674e74e4464e0bfaec16629b5/href">https://medium.com/media/08cbfb1674e74e4464e0bfaec16629b5/href</a></iframe><h4>Aggregate Functions</h4><p>Aggregate functions perform aggregate calculations in the query. When a window is defined the aggregation is restricted within that window. If no window is provided aggregation is performed from the start.</p><p>Below aggregations are supported at the moment.</p><ul><li><strong>avg</strong> : Calculates the average for a given argument for all the events.</li><li><strong>sum</strong> : Returns the sum of a given argument for all the events.</li><li><strong>max</strong> : Returns the maximum value of a given argument for all the events.</li><li><strong>min</strong> : Returns the minimum value of a given argument for all the events.</li><li><strong>count</strong> : Returns the count of all the events.</li><li><strong>distinctCount</strong> : Returns the count of distinct occurrences for a given argument.</li><li><strong>maxForever</strong> : This stores the maximum value for a given attribute throughout the lifetime of the query regardless of any windows in-front.</li><li><strong>minForever</strong> : This stores the minimum value for a given attribute throughout the lifetime of the query regardless of any windows in-front.</li><li><strong>stdDev</strong> : Returns the calculated standard deviation of a given argument for all the events.</li></ul><p><em>Example Query</em></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/70e83bc32b094353fe527f9e83b0e2fc/href">https://medium.com/media/70e83bc32b094353fe527f9e83b0e2fc/href</a></iframe><h4>Group By</h4><p>The group by clause allows you to group the aggregate based on specified attributes.</p><p><em>Sample Query</em></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3ad58944c1a6d15487bac3dcbe9a5c5c/href">https://medium.com/media/3ad58944c1a6d15487bac3dcbe9a5c5c/href</a></iframe><h4>Having</h4><p>The having clause allows you to filter events after processing the select statement.</p><p><em>Sample Query</em></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/51371ea010424cbf524ae94f58a27d15/href">https://medium.com/media/51371ea010424cbf524ae94f58a27d15/href</a></iframe><h4>Order By</h4><p>The order by clause allows you to order the aggregated result in ascending and/or descending order based on specified attributes. By default ordering will be done in ascending manner. User can use descending keyword to order in descending manner.</p><p><em>Sample Query</em></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/49fbde4af929c2ef4f57af37f465d079/href">https://medium.com/media/49fbde4af929c2ef4f57af37f465d079/href</a></iframe><h4>Stream Joins</h4><p>Joins allow you to get a combined result from two streams in real-time based on a specified condition.</p><p>Purpose : Streams are stateless. Therefore, in order to join two streams, they need to be connected to a window so that there is a pool of events that can be used for joining. Joins also accept conditions to join the appropriate events from each stream.</p><p>During the joining process each incoming event of each stream is matched against all the events in the other stream’s window based on the given condition, and the output events are generated for all the matching event pairs.</p><p><strong><em>Unidirectional Join Operation</em></strong></p><p>By default, events arriving at either stream can trigger the joining process. However, if you want to control the join execution, you can add the unidirectional keyword next to a stream in the join definition as depicted in the syntax in order to enable that stream to trigger the join operation. Here, events arriving at other stream only update the window of that stream, and this stream does not trigger the join operation.</p><p>Note : The unidirectional keyword cannot be applied to both the input streams because the default behavior already allows both streams to trigger the join operation.</p><p>Following are the supported operations of a join clause.</p><p><strong><em>Inner join (join)</em></strong></p><p>This is the default behavior of a join operation. join is used as the keyword to join both the streams. The output is generated only if there is a matching event in both the streams.</p><p><strong><em>Left outer join</em></strong></p><p>The left outer join operation allows you to join two streams to be merged based on a condition. left outer join is used as the keyword to join both the streams. Here, it returns all the events of left stream even if there are no matching events in the right stream by having null values for the attributes of the right stream.</p><p><strong><em>Right outer join</em></strong></p><p>This is similar to a left outer join. right outer join is used as the keyword to join both the streams. It returns all the events of the right stream even if there are no matching events in the left stream.</p><p><strong><em>Full outer join</em></strong></p><p>The full outer join combines the results of left outer join and right outer join. full outer join is used as the keyword to join both the streams. Here, output events are generated for each incoming event even if there are no matching events in the other stream.</p><p><em>Example Query</em></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d39dcfbe4de10683a30ee0ebcdda87bc/href">https://medium.com/media/d39dcfbe4de10683a30ee0ebcdda87bc/href</a></iframe><h4>Table Operations with Stream</h4><p>Ballerina provides extensive support to deal with tables. It provides various types of operations such as create, insert, delete and etc.. with in-memory or external storage tables. You could refer below link to find more information on this.</p><ul><li><a href="https://medium.com/@anupama.pathirage/introduction-to-ballerina-tables-205286a53752">https://medium.com/@anupama.pathirage/introduction-to-ballerina-tables-205286a53752</a></li></ul><p>In Streaming context, a table is a stored version of a stream or a table of events. Ballerina provides support to interactively query the state of the stored events in the table when processing events which are arrived through a stream.</p><p><em>Example Query</em></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/0d3efdba2a6a59706b3abb6d9eee74d2/href">https://medium.com/media/0d3efdba2a6a59706b3abb6d9eee74d2/href</a></iframe><h3>Progress So Far</h3><p>As of now, Ballerina based Streaming Implementation provides almost all of the features (as given in the previous section) that required for stream processing. We are currently focusing on below areas as the next step.</p><ul><li>Patterns and Sequences</li><li>State Snapshot Persistence</li></ul><h3>How to Use?</h3><p>Stream processing capabilities which are available in Ballerina added as the <em>experimental</em> feature. Current Ballerina distribution comes with both Siddhi based Streams and native Ballerina based streams implementation. And Ballerina based streaming implementation is enabled by default from Ballerina release 0.990.1.</p><p>If you want to tryout streaming capabilities in Ballerina then run bal files with --experimental flag. For example, if you want to run the ballerina file abcd.bal then run/compile the code with command ballerina run --experimental abcd.bal .</p><p>Please try out and let us know your feedback. Use Ballerina <a href="https://ballerina.io/downloads/">release version 0.990.1</a> or above for this.</p><p>Thank you…</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DuQrJmF0HA7LtijJrNln0w.jpeg" /><figcaption><a href="https://www.flickr.com/photos/jayneandd/4450623309/">https://www.flickr.com/photos/jayneandd/4450623309/</a></figcaption></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=826524c0a6c8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[11 Unbeaten Capabilities of WSO2 Stream Processor]]></title>
            <link>https://mohan-vive.medium.com/11-unbeaten-capabilities-of-wso2-stream-processor-358192d800af?source=rss-eba6b4500bcc------2</link>
            <guid isPermaLink="false">https://medium.com/p/358192d800af</guid>
            <category><![CDATA[cep]]></category>
            <category><![CDATA[wso2]]></category>
            <category><![CDATA[big-data]]></category>
            <category><![CDATA[stream-processing]]></category>
            <category><![CDATA[integration]]></category>
            <dc:creator><![CDATA[Mohanadarshan Vivekanandalingam]]></dc:creator>
            <pubDate>Sun, 09 Dec 2018 03:59:52 GMT</pubDate>
            <atom:updated>2018-12-12T14:01:08.066Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IAMoVVoJ9FeuVT9I43-bAg.jpeg" /><figcaption><a href="https://www.flickr.com/photos/xavier33300/15046006669/">https://www.flickr.com/photos/xavier33300/15046006669/</a></figcaption></figure><p>I hope all of you who are reading through this post well aware about WSO2 Stream Processor. If not, please refer one of my previous <a href="https://medium.com/@mohan.vive/stream-processing-in-enterprise-integration-world-f15989938951">post</a> and <a href="https://wso2.com/analytics-and-stream-processing/">official site</a> for more information. In this blog post, I will be discussing 11 most impressive and prominent capabilities of <a href="https://wso2.com/analytics-and-stream-processing/">WSO2 Stream Processor</a> and its uniqueness compared to other Stream Processor systems in the market.</p><p>WSO2 Stream Processor is a lightweight, open source, high performance, stream processing platform that understands streaming SQL queries in order to capture, analyze, process and act in real time. WSO2 Stream Processor is powered by <a href="https://wso2.github.io/siddhi/">Siddhi</a> CEP engine which is one of the fastest and feature rich CEP engines in the market.</p><h3>Support for Lambda Architecture</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hvXFr6tAq4OegNGBcBJ-yA.png" /><figcaption>High Level View of Lambda Architecture by Dr. <a href="https://medium.com/@srinathperera">Srinath Perera</a></figcaption></figure><p><a href="http://www.databasetube.com/database/big-data-lambda-architecture/">Lambda architecture</a> is one of the common practices used in the big data world. It is an architecture style introduced to provide understanding and to solve concerns involved with the use cases that dealt with both stream processing and batch processing.</p><p>WSO2 Stream Processor highly aligned to the lambda architecture with its capabilities provided by WSO2 Siddhi CEP engine. Event tables and query API features in Siddhi allow users to achieve the lambda needs easily.</p><h4>Event Tables / Data Store</h4><p>Event tables allow Siddhi to work with stored data. Events which arrives in Siddhi engine can join with the stored data to make useful decisions. By defining a schema for tables Siddhi enables them to be processed by queries using their defined attributes with the streaming data. You can also interactively query the state of the stored events in the table.</p><p><strong><em>Sample Query</em></strong></p><p>In the below Siddhi streaming query, we are updating the entries which are in the table when a specific condition met based on the incoming events.</p><pre>define table RoomOccupancyTable (roomNo int, people int);<br>define stream UpdateStream (roomNumber int, arrival int, exit int);<br><br>from UpdateStream<br>select *<br>update RoomOccupancyTable<br>    set RoomOccupancyTable.people = RoomOccupancyTable.people + <br>         arrival - exit<br>    on RoomOccupancyTable.roomNo == roomNumber;</pre><p>As similar to above, we could also join with an event table (which mirrors an external data source such as RDBMS) for the incoming events and make decisions.</p><pre>define table RoomTypeTable (roomNo int, type string);<br>define stream TempStream (deviceID long, roomNo int, temp double);</pre><pre>from TempStream join RoomTypeTable<br>    on RoomTypeTable.roomNo == TempStream.roomNo<br>select deviceID, RoomTypeTable.type as roomType, type, temp<br>    having roomType == &#39;server-room&#39;<br>insert into ServerRoomTempStream;</pre><p>Please refer below references for more details on this.</p><ul><li><a href="https://wso2.github.io/siddhi/documentation/siddhi-4.0/#table">https://wso2.github.io/siddhi/documentation/siddhi-4.0/#table</a></li><li><a href="https://wso2-extensions.github.io/siddhi-store-rdbms">https://wso2-extensions.github.io/siddhi-store-rdbms</a></li></ul><h4>Query API for RDBMS</h4><p><a href="https://wso2-extensions.github.io/siddhi-store-rdbms/api/4.0.45/#query-stream-processo">This</a> function performs SQL retrieval queries on WSO2 data sources which are backed by RDBMS; then, it allows you to query underneath persistence store for the incoming events which arrive in real-time, get the results and performs necessary joins/decision accordingly.</p><p><strong><em>Sample Query</em></strong></p><pre>from TriggerStream#rdbms:query(&#39;SAMPLE_DB&#39;,<br>      &#39;select * from Transactions_Table&#39;, &#39;creditcardno string,<br>           country string, transaction string, amount int&#39;)<br>select creditcardno, country, transaction, amount<br>insert into recordStream;</pre><p>Other than above, you can also perform insertion, deletion and update through <a href="https://wso2-extensions.github.io/siddhi-store-rdbms">CUD processor</a> support provided by Siddhi.</p><h3>Support for Incremental Aggregation</h3><p>Data summarization is one of the common features supported by a stream processing engine. It is divided into two segments; they are summarization over the short time and summarization over a long time.</p><p>Summarization over short time happens using the constructs such as <a href="https://wso2.github.io/siddhi/documentation/siddhi-4.0/#window">windows</a> and <a href="https://wso2.github.io/siddhi/documentation/siddhi-4.0/#aggregate-function">aggregators</a>. You can refer below links for better understanding on that.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zqN_VTzvBSO079AlU7EsAA.png" /><figcaption>Incremental Aggregation Approach in WSO2 Stream Processor</figcaption></figure><p>But, the interesting fact about WSO2 Siddhi is the summarization over a long time duration. It has a very unique and powerful feature <a href="https://docs.wso2.com/pages/viewpage.action?pageId=85365104">incremental aggregation</a> to handle summarization over a long time duration. Incremental aggregation is achieved in WSO2 Stream Processor due to its deep alignment with the Lambda architecture. Incremental aggregation allows you to obtain aggregates in an incremental manner for a specified set of time periods. This not only allows you to calculate aggregations with varied time granularity, but also allows you to access them in an interactive manner for reports, dashboards, and for further processing. Its schema is defined via the aggregation definition.</p><h3>Support for On-demand Stream Processing</h3><p>Usually, Stream Processors are processing events in an asynchronous manner. It receives events from various event sources, processes those events based on the predefined queries which are configured and then alert/notify the output events to the respective parties. In this whole process, event source simply push events to the Stream Processor or Stream Processor pull events from event source and they forgot about the event source and continue to process them; simply there is no any request-response or synchronous behavior.</p><p>But, in the current context, there are use cases that require stream processing with synchronous nature. For example, <a href="https://docs.wso2.com/display/IS570/Using+WSO2+Stream+Processor+for+Adaptive+Authentication">adaptive authentication</a> is one such use case which stream processing is required in a synchronous manner.</p><p>In WSO2 Stream Processor. there are two flavors of on-demand stream processing. They are;</p><ul><li><a href="https://docs.wso2.com/display/SP430/Store+APIs">Store Query API</a> : It allows users to query the data which are in event tables or aggregation by passing simple SQL like query.</li><li><a href="https://wso2-extensions.github.io/siddhi-io-http/">Synchronous HTTP Request-Response</a> : A user can make a request, then the request is converted into an event and processed in synchronous manner and respond back to the user with the result. By this approach, a user can perform a simple filter or even a complex join and etc...</li></ul><p>Please refer below blog post authored by <a href="https://medium.com/@suhothayan">Suhothayan</a> on this. It will provide more details on this.</p><p><a href="https://medium.com/@suhothayan/synchronous-request-response-based-real-time-processing-with-wso2-stream-processor-38613d32a1ec">Synchronous Request-Response based Real-time Processing with WSO2 Stream Processor</a></p><h3>Non Occurrence Pattern Detection</h3><p>Pattern detection is a most common feature supported by a Complex Event Processing engines. In most of these CEP engines, pattern detection is performed only based on the events that are arrives to the CEP engine; which means patterns detection with the absent data (or events not arrived) is cumbersome; but this is one of the common requirement in pattern detection use cases.</p><p>WSO2 Siddhi engine which powers the WSO2 Stream Processor provides extensive support to achieve this. It allows you to write pattern queries to identify a non-occurrence pattern for various use cases. Please refer to <a href="https://wso2.github.io/siddhi/documentation/siddhi-4.0/#detecting-non-occurring-events">documentation link</a> for more details. Above feature contributed to Siddhi by the Google Summer of Code 2017 participant Gobinath. Please refer his <a href="https://medium.com/@lgobinath/gsoc-siddhi-pattern-to-detect-absence-of-events-8b267bd173fe">blog post</a> on this.</p><p><strong><em>Sample Query</em></strong></p><p>Generate an alert if the temperature does not reduce to 12 degrees within 5 minutes of switching on the regulator.</p><pre>define stream RegulatorStateChangeStream<br>(deviceID long, roomNo int, tempSet double, action string);</pre><pre>define stream TempStream (deviceID long, roomNo int, temp double);</pre><pre>from e1=RegulatorStateChangeStream[action == &#39;start&#39;] <br>-&gt; not TempStream[e1.roomNo == roomNo and temp &lt; 12] for &#39;5 min&#39;<br>select e1.roomNo as roomNo<br>insert into AlertStream;</pre><h3>Support for Online Machine Learning</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*r3ywWFiBVTfs7hECWhChQw.jpeg" /><figcaption><a href="https://www.flickr.com/photos/cwkarl/15433742780/">https://www.flickr.com/photos/cwkarl/15433742780/</a></figcaption></figure><p>WSO2 Stream Processor comes with inbuilt support for predictive analytics through machine learning. Usually, in machine learning, historical data is collected to build a predictive model in order to predict the future. However, models can be outdated with time if it is not updated with new data that arrives. WSO2 SP addresses this issue by supporting online machine learning algorithms that enable you to evolve your models while they are alive. A key characteristic of an online predictive algorithm is that the memory and time requirements of the predictive analytics process do not grow over time.</p><p>WSO2 Stream Processor provide <a href="https://docs.wso2.com/display/SP430/Streaming+Machine+Learning">online machine learning (streaming machine learning) </a>support through classification, clustering, and regression. Other than that, online machine learning is also supported through Markov models for anomaly detection.</p><p>Other than above mentioned online machine learning approaches, it also supports conventional machine learning approach which uses pre-built and trained ML models. They are,</p><ul><li><a href="https://wso2-extensions.github.io/siddhi-gpl-execution-pmml/">PMML</a> (build from Python, R, Spark, H2O.ai, etc)</li><li><a href="https://wso2-extensions.github.io/siddhi-execution-tensorflow/">TensorFlow</a></li></ul><h3>Support for Change Data Capture</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GTrAS38vDJkYNe6pw88_fA.png" /><figcaption>CDC Approach with WSO2 Stream Processor</figcaption></figure><p>Change data capture (CDC) become one of the important need in the ETL space. CDC means is a process of capturing the changes that occur in data sources such as RDBMS databases or logs and derive/extract useful relevant data. WO2 Stream Processor provides CDC support from the recent past through a <a href="https://wso2-extensions.github.io/siddhi-io-cdc/">Siddhi extension</a>.</p><p>As of now, respective Siddhi extension only tested with few of the RDBMS types (such as Mysql and Oracle). Using the CDC feature, we could capture the changes that occur in the data sources, send those events through a stream, perform event processing with the pre-defined streaming queries and make a decision which helps business operations and etc…</p><h3>Intelligent Development Studio</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cWEk_eP5NJ3Pu20aJCf8qA.png" /><figcaption>View of WSO2 Stream Processor Development Studio</figcaption></figure><p>WSO2 Stream Processor comes with an intelligent development studio (editor) which supports for both source view and design view. It provides numerous support to build a streaming query (<a href="https://wso2.github.io/siddhi/documentation/siddhi-4.0/#siddhi-application">Siddhi App</a>) in a very short time, test it by simulating events and debug through the queries that implemented. It provides below capabilities.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FU5yYyUTx2bA%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DU5yYyUTx2bA&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FU5yYyUTx2bA%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/b0f22405a9f21ca7cb320f84d6ac999f/href">https://medium.com/media/b0f22405a9f21ca7cb320f84d6ac999f/href</a></iframe><ul><li>Syntax highlighting &amp; auto completion</li><li>Deploy and test streaming queries</li><li>Debugger</li><li>Event simulation</li><li>Sample event generation</li><li>Data explorer for event tables &amp; etc..</li></ul><p>You could refer the video in the left to understand the capabilities of the Development Studio in detail.</p><h3>Support for Citizen Integration</h3><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2F7WGJf4lxjqc%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D7WGJf4lxjqc&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2F7WGJf4lxjqc%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/cd5e1761641b5ded1dca7bdedb612787/href">https://medium.com/media/cd5e1761641b5ded1dca7bdedb612787/href</a></iframe><p>WSO2 Development Studio provides necessary features required for a developer to implement the streaming queries but we have found there is gap/requirement where non-technical users wanted to configure or change existing rules or build some rules using the templates which is already available. Then to cater this requirement WSO2 Stream Processor built the feature called Business Rules Manager for Citizen Integration.</p><p>Business Rules Management dashboard is a user-friendly interface that can be used to build business rules (Siddhi applications customized as per business requirements). WSO2 Stream Processor editor runtime comes with a tool which allows the developer to build such templates that the business can use to build his/her own rules.</p><p>Refer below posts about Business Rules Manager written by <a href="https://medium.com/@senthuran16">Senthuran</a> who is one of the prominent contributor to WSO2 Stream processor.</p><p><a href="https://medium.com/@senthuran16/all-about-the-wso2-sp-business-rules-manager-e7e730b2abe7">All about the WSO2 SP Business Rules Manager</a></p><h3>Dashboard &amp; Reporting Support</h3><p>WSO2 Stream Processor comes with feature-rich dashboard and reporting capabilities. It not only allows users to create gadgets and dashboards but also provide necessary features that required for report generation.</p><h4>Report Generation</h4><p>The Dashboard Portal of WSO2 Stream Processor allows you to <a href="https://docs.wso2.com/display/SP430/Generating+Reports">generate reports</a> with graphical illustrations of processed data. It supports below options for report generation.</p><ul><li>Supported to generate report as CSV and PDF</li><li>Jasper based report generation</li></ul><h4>Dashboard and Gadgets</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vy8urC3a3D6ZeG9aaXrBqg.png" /><figcaption>Sample Dashboards and Gadgets in WSO2 Stream Processor</figcaption></figure><p>The dashboard which is in WSO2 Stream Processor is based on React JS. There are features to create dashboards and widgets through a wizard-based approach; it provides necessary support to design dashboards and widgets as per our need.It also comes with a fine-grained permission model for dashboards, widgets, and data. It also supports localization and inter widget communication for the better interactive dashboard.</p><p>Other than above, it also provides;</p><ul><li>Shareable dashboards with widget state persistence</li><li>Dashboard export and import</li></ul><h3>Status Dashboard</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2A8KnIGvUrkpjLCMGF97Cw.png" /><figcaption>Example View of the Status Dashboard of WSO2 Stream Processor</figcaption></figure><p>A built-in dashboard which helps to monitor WSO2 Stream Processor nodes. This involves monitoring whether all processes of the WSO2 SP setup are working in a healthy manner, monitoring the current status of an SP node, and viewing metrics relating to the history of a node or the cluster. Both JVM level metrics or Siddhi application level metrics can be viewed from the monitoring dashboard.</p><p>A user can configure monitoring for individual Stream Processor node or to a <a href="https://medium.com/@gowthamy/distributed-view-in-wso2-status-dashboard-ef2df1472009">distributed cluster</a>. Through this dashboard, users/admin can find details on below areas.</p><ul><li>Memory usage (Both JVM and Siddhi App level)</li><li>CPU usage (JVM level)</li><li>System load average</li><li>Overall throughput (Node and Siddhi App level)</li><li>Latency (Siddhi App level)</li><li>…</li></ul><p>Other than above, it also provides some metrics such as throughput, memory usage and etc.. at the granularity of Siddhi query level as well.</p><h3>HA Deployment with 100K Throughout</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*h6Ih4iwj4myciLrYiTs5sA.png" /><figcaption>HA Deployment of WSO2 Stream Processor</figcaption></figure><p>As mentioned before WSO2 Stream Processor is powered by WSO2 <a href="https://github.com/wso2/siddhi">Siddhi</a> CEP engine; it is one of the fastest open source CEP engines in the market. With all these, WSO2 Stream Processor able process around <a href="https://docs.wso2.com/display/SP430/Performance+Analysis+Results">100,000 events in a second</a> (for simple queries) with just 2 nodes; while other CEP or Steam Processor engines are required more than 5 nodes to achieve that throughput.</p><p>The <a href="https://docs.wso2.com/display/SP430/Minimum+High+Availability+Deployment">High Availability (HA) deployment </a>is so simple where deployment can be done with a database; it does not require zookeeper, Kafka or any other distributed coordination services. We could also achieve the multi-datacenter requirement with WSO2 Stream Processor HA deployment. The HA deployment is also supported incremental <a href="https://docs.wso2.com/display/SP430/Configuring+Database+and+File+System+State+Persistence">state persistence</a> and recovery as well.</p><p>Thanks for reading the post. Hope I have given enough resources to understand the WSO2 Stream Processor and its prominent capabilities. Please <a href="https://wso2.com/analytics-and-stream-processing/">tryout</a> WSO2 Stream Processor and let us know your feedback.</p><p>Thanks…</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=358192d800af" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Stream Processing in Enterprise Integration World]]></title>
            <link>https://medium.com/siddhi-io/stream-processing-in-enterprise-integration-world-f15989938951?source=rss-eba6b4500bcc------2</link>
            <guid isPermaLink="false">https://medium.com/p/f15989938951</guid>
            <category><![CDATA[stream-processing]]></category>
            <category><![CDATA[integration]]></category>
            <category><![CDATA[wso2]]></category>
            <category><![CDATA[complex-event-processing]]></category>
            <category><![CDATA[ballerina]]></category>
            <dc:creator><![CDATA[Mohanadarshan Vivekanandalingam]]></dc:creator>
            <pubDate>Sun, 18 Nov 2018 16:14:26 GMT</pubDate>
            <atom:updated>2018-11-26T09:05:56.929Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Kh3f_ghCNC7ux6G9PEtwyQ.jpeg" /><figcaption><a href="https://www.flickr.com/photos/kelmon/8559569917/">https://www.flickr.com/photos/kelmon/8559569917/</a></figcaption></figure><p>Some of you might be wondering about the subject and thinking what is the relation between the Stream Processing and Enterprise Integration. Before discussing more details about the subject we need to understand what is meant by Stream Processing, why enterprise integration and then we’ll focus on understanding the relationship between them.</p><h3>What is Stream Processing?</h3><p>I don’t want to bring a totally new interpretation for this. Let me give you the well-accepted definition given by the analysts.</p><p><strong>Forrester</strong> defines <strong>Stream Processing</strong> as the heart of all streaming flows is a sequence of streaming operators that are configured and threaded together to process and analyze the incoming data streams.</p><p>It also defines the <strong>Stream Processing applications</strong> as the software that provides analytical operators to orchestrate data flow, calculate analytics, and detect patterns on event data from multiple, disparate live data sources to allow developers to build applications that sense, think, and act in real time.</p><p><a href="https://www.forrester.com/report/The+Forrester+Wave+Streaming+Analytics+Q3+2017/-/E-RES136545">The Forrester Wave™: Streaming Analytics, Q3 2017</a></p><p><strong>Gartner</strong> defines [2] <strong>Streaming Processing platforms</strong> as the software systems that perform real-time or near-realtime calculations on event data “in motion.” The input is one or more event streams containing data about customer orders, insurance claims, bank deposits/withdrawals, tweets, Facebook postings, emails, financial or other markets, or sensor data from physical assets such as vehicles, mobile devices or machines. The platforms process the input data as it arrives (hence “in motion”), before optionally storing it in some persistent store. They retain a relatively small working set of stream data in memory, just long enough to perform calculations on a set of recent data for the duration of a time window.</p><p><a href="https://www.gartner.com/doc/3879278/market-guide-event-stream-processing">Market Guide for Event Stream Processing</a></p><p>More or like, both analyst provides similar definitions for Stream Processing. In simple words, <strong>Stream Processing is an approach to process incoming events on the fly based on predefined queries.</strong></p><p>You all can refer below article written by Dr. Srinath Perera to get more in-depth knowledge about the Stream Processing and its internals.</p><ul><li><a href="https://medium.com/stream-processing/what-is-stream-processing-1eadfca11b97">https://medium.com/stream-processing/what-is-stream-processing-1eadfca11b97</a></li></ul><h3>Why Enterprise Integration?</h3><p>I am sure, all of you know or aware about integration (If not, why did u looking at the article noh? 😃). But, let’s find what analysts say about Enterprise Integration.</p><p><strong>Gartner</strong> says Enterprise Integration (or Enterprise Application Integration) enable independently designed applications to work together. Key capabilities of application integration technologies include communication functionality that reliably moves messages among endpoints; support for fundamental web and web services standards; functionality that dynamically binds consumer and provider endpoints; message validation, mapping, transformation, and enrichment functionality; orchestration; and support for multiple interaction patterns, content-based routing, and typed messages</p><p>Now, we have understood what analysts say about Stream Processing and enterprise integration; it is time to understand the relation between them.</p><h3>Streaming Data Integration</h3><p>I believe, most of your worked either with stream processing use cases or integration use case or both of them. When you dealt with streaming data in the integration world then it is called <strong>Streaming Data Integration</strong>. Even though concept of streaming data integration is used for a long time, the buzzword <strong>Streaming Data Integration</strong> is evolved in the recent past. This has become one of the common requirement of an Event Stream Processing platform. It is also highlighted in the Gartner report that referred above; it mentioned as Stream Data Integration is one of the two subsegments of the latest event stream processing applications.</p><p>As per <strong>Gartner</strong>, streaming data integration primarily focus on the ingestion and processing of data sources targeting real-time extraction, transformation and loading (ETL) and data integration use cases. The products filter and enrich the data, and optionally calculate time-windowed aggregations before storing the results in a database, file system or some other store such as an event broker. Analytics, dashboards and alerts are a secondary concern for products in this subsegment.</p><h3>Legacy Approach For Streaming Data Integration</h3><p>In the past, this was done by integrating an Integration Server with a Stream Processor/Complex Event Processor . Refer below Figure to get a better understanding of the approach.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/955/1*0QRdhv6_NjnpuegT6T75pQ.jpeg" /><figcaption>Legacy Approach for Streaming Data Integration</figcaption></figure><p>As shown in above Figure, Integration Server or Enterprise Service Bus (ESB) is used to connect with data sources to consume streaming data. Then those events are pass through or minimally processed/transformed events are sent to the Complex Event Processor (CEP) for event processing. In this approach, CEP does the stateful event processing.</p><h3>Latest Approaches For Streaming Data Integration</h3><p>Currently, there are various products or platforms available in the market to achieve this but if we have an open source solution then it would be an added advantage. WSO2 Stream Processor provides extensive capabilities that required for Streaming Data Integration.</p><h4>WSO2 Stream Processor</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*VW2HpNH6-QxlkUxGSUiCnA.png" /><figcaption>Logo of WSO2 Stream Processor</figcaption></figure><p>WSO2 Stream Processor is a lightweight, open source, high performance, stream processing platform that understands streaming SQL queries in order to capture, analyze, process and act in real time. This facilitates real-time, intelligent, actionable business insights. With the product’s simple deployment and its ability to adapt to changes rapidly, enterprises can go to market faster and achieve greater ROI. Unlike other offerings, it also provides high availability, and throughput with just two nodes, along with its distributed deployment. The Siddhi Streaming SQL language also enables users to adapt to the market faster with quicker development times.</p><p>Let’s have a quick look about the Streaming Data Integration related capabilities.</p><ul><li><strong>Extensive connector support to consume or publish events from/to various event sources/sinks : </strong>HTTP, Kafka, TCP, In-memory, WSO2Event, Email, JMS, File, RabbitMQ, MQTT, WebSocket, Amazon SQS andTwitter and allows customization.</li><li><strong>Support to connect with various external data stores : </strong>RDBMS (H2,mysql, ms sql, postgres, oracle and DB2), HBase, Solr, MongoDB, Redis, Hazelcast, Elastic search and Cassandra</li><li><strong>Extensive support for event transformation and manipulation with dedicated feature support : </strong>JSON operations, regex operations, string operations, math operations, time operations and etc..</li><li><strong>Support for Change Data Capture : </strong>WSO2 Stream Processor, provides built-in support for Change Data Capture. At the moment, this support is available through a Siddhi extension.</li></ul><p>You could refer below blog written by Suho (Team Lead of Stream Processor Product) to get more interesting facts about related to ETL on Stream Processor.</p><ul><li><a href="https://medium.com/stream-processing/perform-realtime-and-periodic-etl-with-wso2-stream-processor-358680689935">https://medium.com/stream-processing/perform-realtime-and-periodic-etl-with-wso2-stream-processor-358680689935</a></li></ul><h4>Ballerina</h4><p>Ballerina is a cloud native programming language which designed to provide first class support for integration constructs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/662/0*1yze0Z-R8THh1eKW" /><figcaption>Logo of Ballerina Programming Language</figcaption></figure><p>Ballerina is a compiled, transactional, statically and strongly typed programming language with textual and graphical syntaxes. Ballerina incorporates fundamental concepts of distributed system integration into the language and offers a type safe, concurrent environment to implement microservices with distributed transactions, reliable messaging, stream processing, and workflows. Ballerina is a language designed to be integration simple. Based around the interactions of sequence diagrams, Ballerina has built-in support for common integration patterns. (Source: <a href="https://ballerina.io/">https://ballerina.io</a>)</p><p>Ballerina provides a number of various features that required to make integration simple. Below are some areas that focused by Ballerina.</p><ul><li>API Constructs</li><li>Asynchronous</li><li>Workers</li><li>JSON AND XML</li><li>Annotations</li><li>Streams</li><li>Transactions</li><li>Taint Analysis</li></ul><p>You could refer below link to get a better understanding about the language, its design principles and philosophy.</p><p><a href="https://ballerina.io/philosophy/">Cloud Native Programming Language</a></p><p>As a integration focused language Ballerina clearly understands the need of stream processing in the integration world. Due to this Ballerina provides you the first class support native support for stream processing.</p><p>Now, if you have both integration capabilities and streaming processing capabilities in one place then it would be the ideal thing that can happen. Cool Isn’t it ? 😄</p><p>Now, let’s find a simple use case in the integration space.</p><p><strong>An API which able to serve HTTP requests.</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f39a58265bc7d604cebf3d516fdbe309/href">https://medium.com/media/f39a58265bc7d604cebf3d516fdbe309/href</a></iframe><p>With the few lines of code, you could develop a HTTP service and run it. Now, let’s assume that you wanted to throttle those requests. This requirement can be simply implemented using the stream processing support available in Ballerina.</p><p><strong>Query for a Throttling Requirement</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/477b62c13943472f744c4641da0dd9e2/href">https://medium.com/media/477b62c13943472f744c4641da0dd9e2/href</a></iframe><p>Above is a simple query example to trigger an alert when API requests are more than the expected limit within a time duration. The streaming query simply, listens to the events coming through requestStream and if it identified more than 10 requests from the same host for last 10 seconds then it send an alert by publishing an event to the stream holds the alerts.</p><p>You could find more examples and references on Ballerina Stream processing capabilities in below references.</p><ul><li><a href="https://ballerina.io/learn/ballerina-streaming-reference/">https://ballerina.io/learn/ballerina-streaming-reference/</a></li><li><a href="https://ballerina.io/learn/by-example">https://ballerina.io/learn/by-example</a></li></ul><h3>How to Select the Best Option?</h3><p>Some of you might question why do we have two types of approaches to achieve the same thing. Yes, it is a very important question to raise. The simple answer is <strong>Code vs Configuration.</strong></p><p>It is a choice of the user. Some users like to achieve their need with configurations but some likes to code. It also how it is going to aligned with your existing infrastructure and model. If you are planning to with the layered architecture then choosing WSO2 Stream Processor would be an ideal.</p><p>If you are more towards the micro services approach and love to code then Ballerina would be the ideal approach to achieve your streaming requirements.</p><p><strong><em>Note</em></strong> : <em>Just an head up, Ballerina Streaming constructs are evolving in a rapid manner these days then expect some interesting updates and changes </em>😌<em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f15989938951" width="1" height="1" alt=""><hr><p><a href="https://medium.com/siddhi-io/stream-processing-in-enterprise-integration-world-f15989938951">Stream Processing in Enterprise Integration World</a> was originally published in <a href="https://medium.com/siddhi-io">siddhi-io</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[OSGI White-Board pattern with a sample]]></title>
            <link>https://mohan-vive.medium.com/osgi-white-board-pattern-with-a-sample-abba6a9a991e?source=rss-eba6b4500bcc------2</link>
            <guid isPermaLink="false">https://medium.com/p/abba6a9a991e</guid>
            <category><![CDATA[patterns]]></category>
            <category><![CDATA[java]]></category>
            <category><![CDATA[wso2]]></category>
            <category><![CDATA[osgi]]></category>
            <category><![CDATA[wso2-carbon]]></category>
            <dc:creator><![CDATA[Mohanadarshan Vivekanandalingam]]></dc:creator>
            <pubDate>Sun, 23 Jun 2013 00:00:00 GMT</pubDate>
            <atom:updated>2018-11-25T12:58:40.303Z</atom:updated>
            <content:encoded><![CDATA[<p>White-Board pattern is an important implementation scenario in OSGI programming. I’m writing this blog post because it is hard to find a proper blog post regarding the white-board pattern with an sample. Even some of you might implemented this pattern without knowing this is what means “White-board pattern”. To get more idea about the post it is necessary to have basic knowledge about the <a href="http://wso2.com/library/tutorials/develop-osgi-bundles-using-maven-bundle-plugin">OSGI</a>.</p><p><strong>Why we need a White-Board pattern?</strong></p><p>The whiteboard pattern was invented because the registration of listeners and reacting to their events became a pain in the ass with many (dynamic) dependencies. Especially, because in a component framework you cannot be sure who will be around, which means having to handle all kinds of optionality if you want to be in the driver seat. The original whitepaper is a bit outdated but it shows different examples why, if you analyze what you have to do, the whiteboard is almost always simpler and more flexible. One of the few cases of having your lunch and eating it too … — <strong>By Peter Kriens</strong> (OSGI pioneer and Guru) From <a href="http://pradeepfernando.blogspot.com/2013/02/patterns-in-osgi-programming-white.htmlhttp://">blog</a> by pradeep</p><p>I am going to give the information regarding the White-Board pattern with a sample based on a scenario that i faced.</p><p>Scenario : There is a OSGI bundle called Transport-Adaptor-Core which contains the details about the Transport Adaptors ( ex: jms, email, local &amp; etc). Each transport adaptor is a separate OSGI bundle, These bundles are implemented separately. Here the issue how these Transport-Adaptor bundle gives their information to the Transport-Adaptor-Core bundle. For this there should be a listener in the core which is listening for different transport adaptor implementations, then when a new transport adaptor implementation found Transport-Adaptor-Core bundle can get those implementation info.</p><p>Above scenario can be implemented with two methods. before going into those methods, we’ll see how we can register newly implemented transport adaptor implementation.</p><pre><em>/**<br> * </em><strong><em>@scr.component </em></strong><em>name=&quot;CassandraTransportAdaptorService.component&quot; immediate=&quot;true&quot;<br> */<br><br></em><strong>public class </strong>CassandraTransportAdaptorServiceDS {<br><br>    <strong>protected void </strong>activate(ComponentContext context) {<br><br>        <strong>try </strong>{<br>            TransportAdaptorFactory cassandraTransportAdaptorFactory = <strong>new </strong>CassandraTransportAdaptorFactory();<br>            context.getBundleContext().registerService(TransportAdaptorFactory.<strong>class</strong>.getName(), cassandraTransportAdaptorFactory, <strong>null</strong>);<br>            log.info(<strong>&quot;Successfully deployed the cassandra transport service&quot;</strong>);<br>        } <strong>catch </strong>(RuntimeException e) {<br>            log.error(<strong>&quot;Can not create transport service &quot;</strong>, e);<br>        }<br>    }<br>}</pre><p>Here we have implemented a TransportAdaptorFactory object and registered it in the bundle context.</p><p><strong>Now we’ll see how we can implement the listener using Service Tracker (Service Listener)</strong></p><pre><em>/**<br> * </em><strong><em>@scr.component </em></strong><em>name=&quot;Transport.Adaptor.Tracker.component&quot; immediate=&quot;true&quot;<br> */<br><br></em><strong>public class </strong>TransportAdaptorServiceTracker <strong>implements </strong>Runnable {<br><br>    <strong>private static final </strong>Log <strong><em>log </em></strong>= LogFactory.getLog(TransportAdaptorServiceTracker.<strong>class</strong>);<br>    Thread  <strong>thread</strong>;<br>    ServiceTracker <strong>tracker</strong>;<br><br>    <strong>protected void </strong>activate(ComponentContext context) {<br><br>        <strong>try </strong>{<br>            <strong>tracker </strong>= <strong>new </strong>ServiceTracker(context.getBundleContext(), TransportAdaptorFactory.<strong>class</strong>.getName(),<br>                    <strong>null </strong>);<br>            <strong>tracker</strong>.open();<br>            <strong>thread </strong>= <strong>new </strong>Thread( <strong>this</strong>, <strong>&quot;DisplayManager Whiteboard&quot; </strong>);<br>            <strong>thread</strong>.start();<br>            <strong><em>log</em></strong>.info(<strong>&quot;Transport Adaptor Tracker started&quot;</strong>);<br>        } <strong>catch </strong>(RuntimeException e) {<br>            <strong><em>log</em></strong>.error(<strong>&quot;Can not create transport service &quot;</strong>, e);<br>        }<br>    }<br><br>    @Override<br>    <strong>public void </strong>run() {<br>        Thread   current = Thread.<em>currentThread</em>();<br>        <strong>int  </strong>n = 0;<br>        <strong>while </strong>( current == <strong>thread </strong>) {<br>            Object [] providers = <strong>tracker</strong>.getServices();<br>            <strong>if </strong>( providers != <strong>null </strong>&amp;&amp; providers.<strong>length </strong>&gt; 0 ) {<br>                <strong>if </strong>( n &gt;= providers.<strong>length </strong>)<br>                    n = 0;<br><br>                TransportAdaptorFactory transportAdaptorFactory = (TransportAdaptorFactory) providers[n++];<br>                ((CarbonTransportAdaptorService)TransportAdaptorServiceValueHolder.getCarbonTransportAdaptorService()).registerTransportAdaptor(transportAdaptorFactory.getTransportAdaptor());<br>            }<br><br>            <strong>try </strong>{ wait( 5000 ); } <strong>catch</strong>( InterruptedException e ) {}<br>        }<br>    }<br>}</pre><p>More details about this implementation and White-Board scenario can be found <a href="http://www.osgi.org/wiki/uploads/Links/whiteboard.pdf">here</a>. But this approach is not efficient on my view.</p><p><strong>Listener implementation with declarative services.</strong></p><pre><em>/**<br> * </em><strong><em>@scr.component </em></strong><em>name=&quot;TransportAdaptorServiceTracker.component&quot; immediate=&quot;true&quot;<br> * </em><strong><em>@scr.reference </em></strong><em>name=&quot;transport.adaptor.tracker.service&quot;<br> * interface=&quot;org.wso2.carbon.transport.adaptor.core.TransportAdaptorFactory&quot; cardinality=&quot;0..n&quot;<br> * policy=&quot;dynamic&quot; bind=&quot;setTransportAdaptorService&quot; unbind=&quot;unSetTransportAdaptorService&quot;<br> */<br><br></em><strong>public class </strong>TransportAdaptorServiceTracker {<br><br>    <em>/**<br>     * initialize the Transport Adaptor Manager core service here.<br>     *<br>     * </em><strong><em>@param context<br>     </em></strong><em>*/<br>    </em><strong>protected void </strong>activate(ComponentContext context) {<br><br>        <strong>try </strong>{<br><br>            log.info(<strong>&quot;Successfully deployed the transport adaptor tracker service&quot;</strong>);<br>        } <strong>catch </strong>(RuntimeException e) {<br>            log.error(<strong>&quot;Can not create the transport manager service &quot;</strong>, e);<br>        }<br>    }<br><br>    <strong>protected void </strong>setTransportAdaptorService(TransportAdaptorFactory transportAdaptorFactory) {<br>        ((CarbonTransportAdaptorService)TransportAdaptorServiceValueHolder.getCarbonTransportAdaptorService()).registerTransportAdaptor(transportAdaptorFactory.getTransportAdaptor());<br>    }<br><br>    <strong>protected void </strong>unSetTransportAdaptorService(TransportAdaptorFactory transportAdaptorFactory) {<br><br>    }<br>}</pre><p>Declarative services is the most easiest and efficient way to implement the the White-Board pattern. Here</p><pre><em>/**<br> * </em><strong><em>@scr.component </em></strong><em>name=&quot;TransportAdaptorServiceTracker.component&quot; immediate=&quot;true&quot;<br> * </em><strong><em>@scr.reference </em></strong><em>name=&quot;transport.adaptor.tracker.service&quot;<br> * interface=&quot;org.wso2.carbon.transport.adaptor.core.TransportAdaptorFactory&quot; cardinality=&quot;0..n&quot;<br> * policy=&quot;dynamic&quot; bind=&quot;setTransportAdaptorService&quot; unbind=&quot;unSetTransportAdaptorService&quot;<br> */</em></pre><p>Above annotation declares (that is why it is called declarative services) that there can be 0 to n instances of “TransportAdaptorFactory”. “setTransportAdaptorService” method is triggered when it is found a new implementation of the “TransportAdaptorFactory”. “unSetTransportAdaptorService” is called when removing a “TransportAdaptorFactory” implementation.</p><p>More details about the White-Board pattern can be found in below links.</p><p>[1] <a href="http://pradeepfernando.blogspot.com/2013/02/patterns-in-osgi-programming-white.html">http://pradeepfernando.blogspot.com/2013/02/patterns-in-osgi-programming-white.html</a></p><p>[2] <a href="http://www.osgi.org/javadoc/r4v43/core/org/osgi/framework/ServiceListener.html">http://www.osgi.org/javadoc/r4v43/core/org/osgi/framework/ServiceListener.html</a></p><p>[3] <a href="http://blog.osgi.org/">http://blog.osgi.org/</a></p><p>Acknowledgement : <a href="http://blog.afkham.org/">Azeez</a> &amp; <a href="http://sameera.adahas.org/">Sameera</a></p><p>– THE END –</p><p><em>Originally published at </em><a href="https://mohanadarshan.wordpress.com/2013/06/"><em>mohanadarshan.wordpress.com</em></a><em> on June 23, 2013.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=abba6a9a991e" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>