<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Madhuka Wickramapala on Medium]]></title>
        <description><![CDATA[Stories by Madhuka Wickramapala on Medium]]></description>
        <link>https://medium.com/@madhukaw?source=rss-ce1af468dc07------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/2*c47WtDsDdZjLZVFsuBXyDg.png</url>
            <title>Stories by Madhuka Wickramapala on Medium</title>
            <link>https://medium.com/@madhukaw?source=rss-ce1af468dc07------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 22:37:57 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@madhukaw/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[How to configure, customize and utilize Ballerina logs]]></title>
            <link>https://medium.com/@madhukaw/how-to-configure-customize-and-utilize-ballerina-logs-134d6ea177c0?source=rss-ce1af468dc07------2</link>
            <guid isPermaLink="false">https://medium.com/p/134d6ea177c0</guid>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[software]]></category>
            <category><![CDATA[ballerinalang]]></category>
            <category><![CDATA[logging]]></category>
            <dc:creator><![CDATA[Madhuka Wickramapala]]></dc:creator>
            <pubDate>Fri, 08 Oct 2021 09:20:25 GMT</pubDate>
            <atom:updated>2021-10-08T09:20:25.085Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/760/0*7NcmRgEarcw9hoEk" /></figure><p><a href="https://ballerina.io/?utm_source=me&amp;utm_medium=link&amp;utm_campaign=me_link_Ballerinalog_211007">Ballerina</a>’s built-in logging module is designed to give you critical visibility into your applications with minimal setup. Whether you’re just getting started or already using Ballerina’s <a href="https://github.com/ballerina-platform/module-ballerina-log">logging module</a>, this guide will show you how to configure this module to log all the data you need, customize it to your desired destinations and formats, and utilize your logs to get deeper insights into your Ballerina applications. In this post, you will learn how to:</p><ul><li>Configure the priority level of logs</li><li>Configure the output destination of logs</li><li>Customize the log output in different formats</li><li>Utilize Ballerina logging capabilities to optimize your application</li></ul><blockquote>Note that I am using <a href="https://ballerina.io/downloads/">Ballerina Swan Lake</a> to run the examples in this blog.</blockquote><h3>Configuring the log level</h3><p>The Ballerina log module has four log levels with their priority in descending order as follows.</p><ol><li>ERROR</li><li>WARN</li><li>INFO</li><li>DEBUG</li></ol><p>By default, all logs of the application that are <strong>INFO</strong> level and above are logged to the console. You can change the log level by a configuration through the <strong>Config.toml</strong> file.</p><p>For instance, consider a Ballerina project named “myproject”.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/dc70f66ac835ba8795f2f5da3e5e7775/href">https://medium.com/media/dc70f66ac835ba8795f2f5da3e5e7775/href</a></iframe><p>If you run this, you will see that the ERROR, WARN, and INFO logs are printed to the console.</p><pre>time = 2021–09–23T10:06:34.286+05:30 level = ERROR module = myorg/myproject message = “error log”</pre><pre>time = 2021–09–23T10:06:34.308+05:30 level = WARN module = myorg/myproject message = “warn log”</pre><pre>time = 2021–09–23T10:06:34.309+05:30 level = INFO module = myorg/myproject message = “info log”</pre><p>To change the log level, create a file named <strong>Config.toml</strong> in the same directory that you are running the application and specify the log level. For instance, if you want to change the log level to DEBUG, add the below entry to the file.</p><pre>[ballerina.log]<br>level = &quot;DEBUG&quot;</pre><p>When you run the application you will notice that all the logs of DEBUG level and above are now getting printed to the console.</p><pre>time = 2021-09-23T10:52:04.359+05:30 level = DEBUG module = &quot;myorg/myproject&quot; message = &quot;debug log&quot;</pre><p>The above configures the log level of the entire application. However, Ballerina also supports assigning different log levels to different modules. For example, consider a Ballerina project called “my-project” with two modules “foo” and “bar”.</p><pre>my-project<br>├── Ballerina.toml<br>├── Config.toml<br>├── main.bal<br>└── modules<br>   ├── bar<br>   │   └── bar.bal<br>   └── foo<br>       └── foo.bal</pre><p>Suppose both “foo” and “bar” modules have a function called “hello” as follows.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/0e8967fd7187b3c87ce02b020517d777/href">https://medium.com/media/0e8967fd7187b3c87ce02b020517d777/href</a></iframe><p>The following is the main function of the default package.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/fd2a4e902e50efc5192af991d7205892/href">https://medium.com/media/fd2a4e902e50efc5192af991d7205892/href</a></iframe><p>When you run this project, you will notice that all the logs of <strong>INFO</strong> level and above in the entire application are getting printed to the console.</p><pre>time = 2021-09-23T11:40:32.774+05:30 level = ERROR module = myorg/myproject message = &quot;error log&quot;</pre><pre>time = 2021-09-23T11:40:32.798+05:30 level = WARN module = myorg/myproject message = &quot;warn log&quot;</pre><pre>time = 2021-09-23T11:40:32.799+05:30 level = INFO module = myorg/myproject message = &quot;info log&quot;</pre><pre>time = 2021-09-23T11:40:32.801+05:30 level = ERROR module = myorg/myproject.foo message = &quot;error log&quot;</pre><pre>time = 2021-09-23T11:40:32.803+05:30 level = WARN module = myorg/myproject.foo message = &quot;warn log&quot;</pre><pre>time = 2021-09-23T11:40:32.810+05:30 level = INFO module = myorg/myproject.foo message = &quot;info log&quot;</pre><pre>time = 2021-09-23T11:40:32.812+05:30 level = ERROR module = myorg/myproject.bar message = &quot;error log&quot;</pre><pre>time = 2021-09-23T11:40:32.813+05:30 level = WARN module = myorg/myproject.bar message = &quot;warn log&quot;</pre><pre>time = 2021-09-23T11:40:32.814+05:30 level = INFO module = myorg/myproject.bar message = &quot;info log&quot;</pre><p>However, you might want to assign different log levels to different modules. Suppose you want to set DEBUG level to the “foo” module, the ERROR level to the “bar” module, and WARN level to the rest of the application. You can achieve this by updating the <strong>Config.toml</strong> file as follows.</p><pre>[ballerina.log]<br>level = &quot;WARN&quot;</pre><pre>[[ballerina.log.modules]]<br>name = &quot;myorg/myproject.foo&quot;<br>level = &quot;DEBUG&quot;</pre><pre>[[ballerina.log.modules]]<br>name = &quot;myorg/myproject.bar&quot;<br>level = &quot;ERROR&quot;</pre><p>Now you will notice that in the “foo” module logs of DEBUG level and above are printed while in the “bar” module only ERROR level logs are printed and in the rest of the application, all the logs of WARN level and above are printed to the console.</p><h3>Customizing the log output in JSON format</h3><p>Ballerina logs are printed in <strong>LOGFMT</strong> format by default.</p><pre>time = 2021-09-23T12:10:57.162+05:30 level = INFO module = myorg/myproject.foo message = &quot;info log&quot;</pre><p>However, you can change the format to <strong>JSON</strong> by placing the below entry in the <strong>Config.toml</strong> file.</p><pre>[ballerina.log]<br>format = &quot;json&quot;</pre><p>Now all the logs of the application will be printed in <strong>JSON</strong> format.</p><pre>{&quot;time&quot;:&quot;2021-09-23T14:05:14.639+05:30&quot;, &quot;level&quot;:&quot;INFO&quot;, &quot;module&quot;:&quot;myorg/myproject.foo&quot;, &quot;message&quot;:&quot;info log&quot;}</pre><h3>Ballerina logging best practices</h3><h4>Making use of key-value pairs</h4><p>The Ballerina log module provides some features that can be used to improve your logging experience. One of them is the ability to log any number of <strong>key/value</strong> pairs. The keys are of <strong>string</strong> type and values can be of <strong>anydata</strong> type including <strong>string, int, float, boolean</strong>, etc. With this feature, you can capture all the important information related to an application event in one place without having the hassle of concatenating multiple parameters into a single message.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/254ced9823f00b4a9c3b8ab9180a3d65/href">https://medium.com/media/254ced9823f00b4a9c3b8ab9180a3d65/href</a></iframe><p>This would print the following log.</p><pre>time = 2021-09-23T14:38:25.178+05:30 level = INFO module = &quot;&quot; message = &quot;info log&quot; id = 845315 name = &quot;foo&quot; successful = true</pre><blockquote>Note that the time printed in the log message is the current local time in <a href="https://datatracker.ietf.org/doc/html/rfc3339">RFC3339</a> format.</blockquote><h4>Print errors as convenient to you</h4><p>Your application might have many occurrences where errors are generated and you may want a quick and simple way to log them. All the print functions in the Ballerina log module accept an error as a separate optional parameter.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/81bb8b287a31bb4f469242fe769fed33/href">https://medium.com/media/81bb8b287a31bb4f469242fe769fed33/href</a></iframe><p>This way you do not have to concatenate the error to the log message and it is easier to identify the error when going through the logs.</p><pre>time = 2021-09-23T19:04:04.289+05:30 level = ERROR module = &quot;&quot; message = &quot;failed to initialize server&quot; error = &quot;port already in use&quot;</pre><p>In some cases, you might want to elaborate on the error and print the complete error stack trace so that it is more convenient for debugging purposes. You can do this by passing the error stack trace as a <strong>key-value</strong> pair.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6f097af43bd22f2ad27cf7270f945cfb/href">https://medium.com/media/6f097af43bd22f2ad27cf7270f945cfb/href</a></iframe><pre>time = 2021-10-06T14:33:08.391+05:30 level = ERROR module = &quot;&quot; message = &quot;error log&quot; stackTrace = [{&quot;callableName&quot;:&quot;f3&quot;,&quot;fileName&quot;:&quot;stacktrace.bal&quot;,&quot;lineNumber&quot;:17},{&quot;callableName&quot;:&quot;f2&quot;,&quot;fileName&quot;:&quot;stacktrace.bal&quot;,&quot;lineNumber&quot;:13},{&quot;callableName&quot;:&quot;f1&quot;,&quot;fileName&quot;:&quot;stacktrace.bal&quot;,&quot;lineNumber&quot;:9},{&quot;callableName&quot;:&quot;main&quot;,&quot;fileName&quot;:&quot;stacktrace.bal&quot;,&quot;lineNumber&quot;:4}]</pre><h4>Passing function pointers to improve the performance</h4><p>Suppose there is a DEBUG log that logs a value returned from a function with heavy computations and the log level is INFO.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ce20da5b2ec6c55168254daa66ac522f/href">https://medium.com/media/ce20da5b2ec6c55168254daa66ac522f/href</a></iframe><p>Even when the DEBUG logs are not printed (since the log level is INFO) the application would still have to execute the function with heavy computations. However, you can avoid this by passing a pointer to the function as a value in a <strong>key/value</strong> pair like follows.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3836c49b948a2f82c79a5d7ddf3dee78/href">https://medium.com/media/3836c49b948a2f82c79a5d7ddf3dee78/href</a></iframe><p>Since the DEBUG level is not enabled, the log:printDebug function will not be called, thus the function with heavy computations will not be executed. This could improve the performance of the application significantly if there is a high number of such functions.</p><h3>Configuring the output destination of logs</h3><p>Sometimes you may want to write the log output to a file instead of printing them to the console. For instance, for a command-line application, it would not be acceptable to spew all its logging information onto the console by default. In such situations, Ballerina supports writing the logs to a file and not printing the logs to the console. To achieve this, you can call the log:setOutputFile function and provide the path of the file. eg:</p><pre>var result = log:setOutputFile(&quot;./resources/myproject.log&quot;);</pre><blockquote>Note that the given file should have a <strong>“.log”</strong> extension.</blockquote><p>Now all the subsequent logs will be written to this file instead of printing to the console. If the given file already exists, Ballerina will append the new logs to this existing file. Otherwise, a file will get created in the given path. However, you can also overwrite the content of the file if the given file already exists. To do that give an additional parameter called log:OVERWRITE to the above function. eg:</p><pre>var result = log:setOutputFile(&quot;./resources/myproject.log&quot;, log:OVERWRITE);</pre><p>Now every time you run the application the existing content of the file will get cleared and the subsequent logs of your application will get written to the file.</p><p>If you want to learn more about Ballerina logging and other functionalities refer<a href="https://ballerina.io/learn/by-example/"> Ballerina by examples</a>. Hope you found this useful!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=134d6ea177c0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deploying extensive applications in Kubernetes using Cellery]]></title>
            <link>https://medium.com/wso2-cellery/deploying-extensive-applications-in-kubernetes-using-cellery-fa7a0258e109?source=rss-ce1af468dc07------2</link>
            <guid isPermaLink="false">https://medium.com/p/fa7a0258e109</guid>
            <category><![CDATA[microservices-application]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[deploy]]></category>
            <category><![CDATA[microservices]]></category>
            <category><![CDATA[cellery]]></category>
            <dc:creator><![CDATA[Madhuka Wickramapala]]></dc:creator>
            <pubDate>Sat, 02 Nov 2019 08:58:28 GMT</pubDate>
            <atom:updated>2019-11-02T08:58:28.322Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7qF9ltMhCZ_tryhEpLsXbg.jpeg" /></figure><h3>Prerequisites</h3><p>Before we get started you need to have <a href="https://github.com/wso2-cellery/sdk/releases/tag/v0.3.0">Cellery</a> installed in your machine and also have a Cellery runtime setup.</p><h3>Introduction</h3><p>Cellery provides a developer runtime, tooling and a management plane which allows developers to run and manage their composite microservice applications on Kubernetes. In this blog, we will discuss how developers can utilize useful capabilities of Cellery when deploying large scale applications.</p><p>First clone Cellery <a href="https://github.com/wso2-cellery/sdk.git">sdk</a> repo and go to test-cases/employee-portal directory. There is a sample hr application with 3 cell instances. The root cell instance is hr and it has 2 dependent cell instances; employee and stock.</p><p>Following is the cell file of hr cell instance and it defines dependencies to employee and stock cell instances; employeeCellDep and stockCellDep.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/10573676196b144f58325d8ef7cd4a51/href">https://medium.com/media/10573676196b144f58325d8ef7cd4a51/href</a></iframe><p>Lets assume the 3 cell images created by the user are; hr : myorg/hr:1.0.0 ,employee : myorg/employee:1.0.0,stock : myorg/employee:1.0.0 . First run employee cell instance as described in the <a href="https://github.com/wso2-cellery/sdk/tree/master/test-cases/employee-portal">readme</a>.</p><p>Now only employee cell instance is running and stock and hr employee instances.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/321/1*t9LfnZYbc7XT0yk6yM70Yw.png" /><figcaption>hr instance with primary dependencies</figcaption></figure><p>Let’s suppose you need to run hr instance but the employee instance is already running. In this scenario, cellery provides a way to run hr instance by providing a link to the already running employee instance. Execute cellery run myorg/hr:1.0.0 -l employeeCellDep:employee -l stockCellDep:stock -d .</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JhrI8i3tOIY14R1a9NaZyg.png" /><figcaption>running hr cell instance</figcaption></figure><p>Here, the link to already running employee cell instance is given by the relation -l alias:instance. Since we provide the flag -d (start-dependencies) the stock instance which is also dependent will get started.</p><p>Now lets further improve the hr instance to have transitive dependent cell instance (job and education).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/446/1*Btsg6DclMDKJ9qnaYo2N4w.png" /><figcaption>hr instance with transitive dependencies and shared dependencies</figcaption></figure><p>In this example we have added two transitive dependencies; “education”: a dependent instance of employee and “job”: shared dependent instance of employee and stock instance. Simply execute cellery run myorg/hr:1.0.0 -n hr -d -s to start the hr instance. Here, by giving the flag -s (share-dependencies) only one instance for of job cell will get started.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1XbBNdBp6Lbsv1knD5eLpQ.png" /><figcaption>running hr instance with shared dependencies</figcaption></figure><p>Note that the instance “job100j8s4” gets marked as “shared” in the table above which means it is shared by both employee cell and stock cell. Furthermore, if the links to the dependent cell instances were not provided, random instance names will be generated as seen above.</p><p>Users can also provide environment variables for each cell instance in the dependency tree. Simply give -e instance.ENV_VAR=VALUE . For instance, if you need to set “host” name for employee instance as “abc”, execute cellery run myorg/hr:1.0.0 -n hr -l stockCellDep:stock -l employeeCellDep:employee -e employee.host=abc -d . This environment variable can then be used in your application. This is important because multiple instances can have environment variables the same name but different values. For example, there can be another environment variable “host” for the instance employee with the value “xyz”. In that case we can provide that environment variable as -e employee.host=xyz.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fa7a0258e109" width="1" height="1" alt=""><hr><p><a href="https://medium.com/wso2-cellery/deploying-extensive-applications-in-kubernetes-using-cellery-fa7a0258e109">Deploying extensive applications in Kubernetes using Cellery</a> was originally published in <a href="https://medium.com/wso2-cellery">wso2-cellery</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Setting up Cellery on Docker-For-Mac]]></title>
            <link>https://medium.com/@madhukaw/setting-up-cellery-on-docker-for-mac-f47f867ca064?source=rss-ce1af468dc07------2</link>
            <guid isPermaLink="false">https://medium.com/p/f47f867ca064</guid>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[cellery]]></category>
            <category><![CDATA[docker-for-mac]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[microservices]]></category>
            <dc:creator><![CDATA[Madhuka Wickramapala]]></dc:creator>
            <pubDate>Sun, 18 Aug 2019 08:25:40 GMT</pubDate>
            <atom:updated>2019-08-18T08:25:40.951Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*Qzm9Fnncs-esMMWdmIVKYA.jpeg" /></figure><h3>Prerequisites</h3><p>Before setting up cellery you need to have <a href="https://docs.docker.com/docker-for-mac/install/">docker-for-mac</a> installed in your machine. You also need to have installed <a href="https://github.com/wso2-cellery/sdk/releases/tag/v0.3.0">Cellery</a>.</p><h3>Introduction</h3><p>Cellery provides a developer runtime, tooling and a management plane which allows developers to run and manage their composite microservice applications on kubernetes. Docker-for-mac is one of the platforms which cellery can be installed onto.</p><h3>Configuring docker-for-mac</h3><p>It is important to note that users need to increase the resources in order to support cellery runtime on docker-for-mac. Minimum resources required are as follows.</p><ul><li>CPU : 4 core or more</li><li>Memory : 8 GB or more</li></ul><p>Go to Preferences &gt; Advanced tab and increase the resources as shown below. Then click on “Apply and Restart” button.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/986/1*wfXJxAW1CXNrzeJFDsSeWw.png" /><figcaption>Increasing resources in docker-for-mac</figcaption></figure><p>If the users want to install Cellery with persisted volume they need to create <strong>/var/tmp/cellery</strong> directory and add it to docker for desktop file sharing to support persistent deployments. For that go to Preferences &gt; File Sharing and click the “+” button and give the path to <strong>/var/tmp/cellery </strong>as shown below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/990/1*aSRV3uwHIEHnVHmwSRTW9A.png" /><figcaption>Mounting volumes in docker-for-mac</figcaption></figure><h3>Installing Cellery on Docker-For-Mac</h3><p>Now everything is ready to install cellery onto Docker-For-Mac and there are two methods : interactive and non-interactive mode.</p><h4>Interactive method</h4><p>Execute cellery setup and select create &gt; Existing cluster. Then select Persistent volume or Non-persistent volume. If Persistent volume is selected users can opt to use NFS server or if not use local volume. Then select basic or complete runtime and finally select “Load balancer” as the ingress mode (since we are using docker-for-mac).This will create a cellery installation on Docker-For-Mac.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/740/1*AmFHlOnOW48qZxs-AaZ0Pw.png" /><figcaption>Installing Cellery via interactive method</figcaption></figure><h4>Non-interactive method</h4><p>Based on user’s requirements; persistent volume/volatile, Have access to NFS server/not and basic/complete execute appropriate cellery inline command to create cellery installation on Docker-For-Mac.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lXii19EQxaucYyrCbuR_Jw.png" /><figcaption>Installing cellery via inline command</figcaption></figure><h3>Configuring host entries</h3><p>Update the /etc/hosts file with localhost (127.0.0.1) to access the system components exposed by cellery.</p><pre>127.0.0.1 wso2-apim cellery-dashboard wso2sp-observability-api wso2-apim-gateway cellery-k8s-metrics idp.cellery-system</pre><h3>Removing Cellery runtime from Docker-For-Mac</h3><p>Users can remove the created cellery runtime from docker-for-mac via Cellery CLI.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/614/1*qRvouQRpLuw9Epd3IgdFSA.png" /><figcaption>Removing cellery via interactive method</figcaption></figure><h3>Interactive method</h3><p>Execute cellery setup and select Manage &gt; Existing Cluster&gt; cleanup. This will remove Cellery installation from docker-for-mac.</p><h3>Non-interactive method</h3><p>To remove cellery runtime in non-interactive mode execute cellery setup cleanup existing -y.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f47f867ca064" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Setting up Cellery on GCP]]></title>
            <link>https://medium.com/wso2-cellery/setting-up-cellery-on-gcp-d5c9bd632506?source=rss-ce1af468dc07------2</link>
            <guid isPermaLink="false">https://medium.com/p/d5c9bd632506</guid>
            <category><![CDATA[google-cloud-platform]]></category>
            <category><![CDATA[cellery]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[cloud]]></category>
            <category><![CDATA[gcp]]></category>
            <dc:creator><![CDATA[Madhuka Wickramapala]]></dc:creator>
            <pubDate>Thu, 15 Aug 2019 10:34:38 GMT</pubDate>
            <atom:updated>2019-08-15T15:29:25.172Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/650/1*pGG2uK1Ug7l4sLM3jtg0_A.png" /></figure><h3>Prerequisites</h3><p>Before setting up cellery runtime you need to have a <a href="https://cloud.google.com/iam/docs/creating-managing-service-accounts">GCP account</a> created and <a href="https://cloud.google.com/sdk/docs/">Gloud SDK</a> installed in your machine. You also need to have installed <a href="https://github.com/wso2-cellery/sdk/releases/tag/v0.3.0">Cellery</a>.</p><h3>Introduction</h3><p>Cellery provides a developer runtime, tooling and a management plane which allows developers to run and manage their composite microservice applications on kubernetes. GCP is one of the platforms which cellery can be installed onto.</p><h3>Configuring GCP project</h3><p>If you have not already created a project, use gcloud init command to create a GCP project. It is important to note that gcloud must be configured to the project in which you want to install cellery runtime onto. Also make sure that <a href="https://cloud.google.com/billing/docs/how-to/manage-billing-account">billing</a> is enabled for that project.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/593/1*GHrgKin617AM2qkpDHCIkA.png" /><figcaption>GCP project creation</figcaption></figure><p>Here I have created a gcp project named “vick-team” and set the region as “us-west1” and the zone as “us-west1-b”. Make sure that region and zone are set correctly. Execute gcloud config list --format json to get the region and zone details.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/521/1*iKYAutabOyWzSHibvappfQ.png" /><figcaption>Gcloud config list</figcaption></figure><p>If region or zone is not set properly go to <a href="https://console.cloud.google.com/compute/settings?_ga=2.20830648.-1274616255.1554447482&amp;project=vick-team&amp;folder&amp;organizationId=732283690743">GCP console</a> and select default region and zone for your project. You can also use the gcloud cli to set the project zone and region.</p><pre>gcloud config set project &lt;project name&gt;<br>gcloud config set compute/region &lt;region&gt;<br>gcloud config set compute/zone &lt;zone&gt; </pre><h3>Enabling GCP Apis needed for Cellery</h3><p>We need to enable some GCP Apis which will be used to create resources in GCP. Go to GCP console then select APIs and Services from menu pane and select Dashboard as shown below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/399/1*iGbBx0N5HUmKyrqsXmarNA.png" /><figcaption>Enabling GCP Apis</figcaption></figure><p>Next click on ENABLE APIS AND SERVICES button at the top and enable “Cloud Engine API”, “Kubernetes API”, “Cloud Filestore API” and “Cloud SQL Admin API”. Alternatively you can enable those Apis by executing below command.</p><pre>gcloud services enable container.googleapis.com file.googleapis.com sqladmin.googleapis.com </pre><p>An Api key is needed to access the above mentioned Apis. Go to IAM and admin from the left menu and select “Service accounts” as shown below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/470/1*tEl-zh0SIjryFqhlc-mP0g.png" /><figcaption>Generating Api key (1)</figcaption></figure><p>Then select the default service account (In this case “<a href="https://console.cloud.google.com/iam-admin/serviceaccounts/details/113266968681123022830?organizationId=732283690743&amp;project=vick-team">vick-team</a>”)&gt; Create Key &gt; JSON option, and download the JSON file.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/999/1*0VLaHCrTj4PWX4Ggqs9SIw.png" /><figcaption>Generating Api key (2)</figcaption></figure><p>Copy the generated json file into $HOME/.cellery/gcp folder .</p><h3>Installing Cellery on GCP</h3><p>Now everything is ready to install cellery onto GCP and there are two methods : interactive and non-interactive mode.</p><h4>Interactive method</h4><p>Execute cellery setup and select Create &gt; GCP &gt; basic or complete. This will create a cellery installation on GCP.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/576/1*aY9iL4PdGGIBJd3LXaUZGA.png" /><figcaption>Installing cellery via interactive method</figcaption></figure><h4>Non-interactive method</h4><p>To create a basic cellery runtime in non-interactive mode execute cellery setup create gcp and execute cellery setup create gcp --complete -y to create a complete cellery runtime.</p><h3>Configuring host entries</h3><p>Execute kubectl get ingress -n cellery-system and get the nginx ip address. Then update the /etc/hosts file with that ip address to access the system components exposed by cellery.</p><pre>&lt;IP Address&gt; wso2-apim cellery-dashboard wso2sp-observability-api wso2-apim-gateway cellery-k8s-metrics idp.cellery-system </pre><h3>Removing Cellery runtime from GCP</h3><p>Users can remove the created cellery runtime from GCP via Cellery CLI.</p><h4>Interactive method</h4><p>Execute cellery setup and select Manage &gt; GCP&gt; cleanup and select the GCP cluster name. This will remove the GCP cluster as well as all the resources; Sql instance, File store and storage.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/578/1*Cgnxep01qM_rlKFVLWm_cA.png" /><figcaption>Removing cellery via interactive method</figcaption></figure><h4>Non-interactive method</h4><p>To remove cellery runtime in non-interactive mode execute cellery setup cleanup gcp &lt;CLUSTER_NAME&gt; .</p><p>In the next article we will discuss how to setup cellery on Docker-For-Mac.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d5c9bd632506" width="1" height="1" alt=""><hr><p><a href="https://medium.com/wso2-cellery/setting-up-cellery-on-gcp-d5c9bd632506">Setting up Cellery on GCP</a> was originally published in <a href="https://medium.com/wso2-cellery">wso2-cellery</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Download files from AWS using go language]]></title>
            <link>https://medium.com/@madhukaw/download-files-from-aws-using-go-language-6f61bc75880a?source=rss-ce1af468dc07------2</link>
            <guid isPermaLink="false">https://medium.com/p/6f61bc75880a</guid>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[s3-bucket]]></category>
            <category><![CDATA[go]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[amazon]]></category>
            <dc:creator><![CDATA[Madhuka Wickramapala]]></dc:creator>
            <pubDate>Sun, 07 Jul 2019 16:47:56 GMT</pubDate>
            <atom:updated>2019-07-07T16:47:56.482Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3PTrJei3zFAylve2qnUglg.png" /></figure><h3>Prerequisites</h3><p>I presume by now you have an AWS account and have an S3 bucket created in it. It is important to give public access to that bucket. If you have not already done that follow <a href="https://havecamerawilltravel.com/photographer/how-allow-public-access-amazon-bucket/">this</a> simple guide and make your s3 bucket public and upload a file onto it. I have used go language to implement so it is necessary for you to have <a href="https://golang.org/dl/">go</a> installed.</p><p>Now that we have all the infrastructure ready we can go ahead and start implementing a function to download the file using go language. We need to give 3 function parameters: name of the bucket, name of the file and the path where the file should be downloaded into.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/80999d7418621325bf966062f35d91ab/href">https://medium.com/media/80999d7418621325bf966062f35d91ab/href</a></iframe><p>As you can see there is another parameter called “displayProgressBar” (Boolean) which will determine whether or not to display the download progress bar and we will look into it later. First a file will be created in the given location.</p><pre>file, err := os.Create(filepath.Join(path, item))</pre><p>Next we need to create a session and give the region and credentials. For me the region is “ap-south-1”. It is important to give “credentials.AnonymousCredentials” as the credentials since our s3 bucket is public. Otherwise users will not be able to download that item.</p><pre>sess, _ := session.NewSession(&amp;aws.Config{<br>   Region: aws.String(constants.<em>AWS_REGION</em>), Credentials: credentials.AnonymousCredentials},<br>)</pre><p>Next obtain the size of the file which would serve 2 purposes. One is to display it to the user and get his confirmation as follows. This will improve the user experience since he get to know the size of the file beforehand.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NIoJW6fDGzpxTZlN7yKkNg.png" /><figcaption>Download confirmation</figcaption></figure><p>The other purpose is to configure the download progress bar. Following is simple function to get the size of the file.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4170e53b4c37daa5466efcdaf15a17a3/href">https://medium.com/media/4170e53b4c37daa5466efcdaf15a17a3/href</a></iframe><p>Next create a download manager and configure it with the number of concurrent downloads and size of each part. Select the optimum configuration that suits you.</p><pre>downloader := s3manager.NewDownloader(sess, <strong>func</strong>(d *s3manager.Downloader) {<br>   d.PartSize = 64 * 1024 * 1024 // 64MB per part<br>   d.Concurrency = 6<br>})</pre><p>Next create a progress bar which would show the progress of the download. This is helpful since it displays important information such as download percentage, download speed and ETA for download completion.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nrdYSUlc3jJAU0Ed_nLxGw.png" /><figcaption>Download progress bar</figcaption></figure><p>To implement the progress bar I have used <a href="https://github.com/cheggaaa/pb/tree/v1.0.28">this</a> go library. ProgresWriter is a struct that encapsulates data related to the downloading process.</p><pre><strong>type </strong>progressWriter <strong>struct </strong>{<br>   writer  io.WriterAt<br>   size    int64<br>   bar     *pb.ProgressBar<br>   display bool<br>}</pre><p>Lets have a look at how to initialize the progressWriter.</p><pre>writer := &amp;progressWriter{writer: file, size: s3ObjectSize}<br>writer.display = displayProgressBar<br>writer.init(s3ObjectSize)</pre><p>Note that the “writer” is the file in which the downloaded data will be written into. The size is the file size of the s3 bucket item derived earlier. Set “displayProgressBar” to display the progress bar while the download happens. For small files we can disable that option by setting “displayProgressBar” false since it is not necessary. Next initialize the progressWriter by executing “init” function and passing the file size.</p><pre><strong>func </strong>(pw *progressWriter) init(s3ObjectSize int64) {<br>   <strong>if </strong>pw.display {<br>      pw.bar = pb.StartNew(int(s3ObjectSize))<br>      pw.bar.ShowSpeed = <em>true<br>      </em>pw.bar.Format(&quot;[=&gt;_]&quot;)<br>      pw.bar.SetUnits(pb.<em>U_BYTES_DEC</em>)<br>   }<br>}</pre><p>This function will start a new progress bar. You can configure it for your liking by setting the display format, unit of downloads and whether or not to show the download speed. Next start the down-loader.</p><pre>numBytes, err := downloader.Download(writer,<br>   &amp;s3.GetObjectInput{<br>      Bucket: aws.String(bucket),<br>      Key:    aws.String(item),<br>   })</pre><p>The “WriteAt” function writes the downloaded data to the file as well as increment the progress bar.</p><pre><strong>func </strong>(pw *progressWriter) WriteAt(p []byte, off int64) (int, error) {<br>   <strong>if </strong>pw.display {<br>      pw.bar.Add64(int64(len(p)))<br>   }<br>   <strong>return </strong>pw.writer.WriteAt(p, off)<br>}</pre><p>Finally after the download is completed we can call the finish function which would stop the progress bar.</p><pre><strong>func </strong>(pw *progressWriter) finish() {<br>   <strong>if </strong>pw.display {<br>      pw.bar.Finish()<br>   }<br>}</pre><p>I hope this helped you in implementing a download process from AWS with a progress bar using go language. You can get access to the full code base from <a href="https://github.com/MadhukaHarith92">here</a>. Thanks.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6f61bc75880a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[[Ballerina] Automated github stars tracker with email notification]]></title>
            <link>https://medium.com/@madhukaw/ballerina-daily-mail-notification-on-github-stars-data-7c705adb8441?source=rss-ce1af468dc07------2</link>
            <guid isPermaLink="false">https://medium.com/p/7c705adb8441</guid>
            <category><![CDATA[wso2]]></category>
            <category><![CDATA[gmail]]></category>
            <category><![CDATA[http-request]]></category>
            <category><![CDATA[github]]></category>
            <category><![CDATA[ballerina]]></category>
            <dc:creator><![CDATA[Madhuka Wickramapala]]></dc:creator>
            <pubDate>Thu, 04 Oct 2018 17:15:03 GMT</pubDate>
            <atom:updated>2018-12-01T14:40:06.945Z</atom:updated>
            <content:encoded><![CDATA[<h4>With the integration of Github API and Gmail API</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/980/1*fJSHiMash23udxsfq7gw3A.jpeg" /></figure><p><strong>Description</strong></p><p>Github has several repositories. The objective of this application is to identify which Github users of a company have not starred particular repositories. There will be a database table with entries of employee GitHub IDs. A daily task will be scheduled to send email notifications to a mail list with a list of Github Ids of employees who have not starred the repository.</p><p><strong>Prerequisites</strong></p><p>Ballerina 0.982.0 or higher version.</p><p><strong>Configuring the system</strong></p><ul><li>First you need to set up Ballerina runtime. You can refer <a href="https://github.com/ballerina-lang/ballerina/blob/master/docs/quick-tour.md">https://github.com/ballerina-lang/ballerina/blob/master/docs/quick-tour.md</a> for further help.</li><li>Next you need to download a mysql connector (Eg: <a href="https://dev.mysql.com/downloads/connector/j/8.0.html">https://dev.mysql.com/downloads/connector/j/8.0.html</a>) and copy the MySQL JDBC driver to the BALLERINA_HOME/bre/lib folder and change the DB connection properties as required. You can refer <a href="https://ballerina.io/learn/by-example/mysql-client.html">https://ballerina.io/learn/by-example/mysql-client.html</a> for further help.</li></ul><p><strong>Use case scenario explained</strong></p><p>Suppose we have a database table with some Github Ids as follows.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/988/1*4OoZWZnGxYxRM6X-6dA36Q.png" /></figure><p>Lets say there are 3 github repositories :</p><ol><li><a href="https://github.com/wso2/siddhi,https://github.com/wso2/msf4j,https://github.com/ballerina-platform/ballerina-lang">https://github.com/wso2/siddhi</a></li><li><a href="https://github.com/wso2/siddhi,https://github.com/wso2/msf4j,https://github.com/ballerina-platform/ballerina-lang">https://github.com/wso2/msf4j</a></li><li><a href="https://github.com/wso2/siddhi,https://github.com/wso2/msf4j,https://github.com/ballerina-platform/ballerina-lang">https://github.com/ballerina-platform/ballerina-lang</a></li></ol><p>and we want to know which employees have not starred these repositories and send an email notification to a mailing list with their GitHub ids. Following is a step by step process.</p><ol><li><strong>Get the GutHub Ids as a string array.</strong></li></ol><p>We need to define the database configuration in a file named ballerina.conf. Then we can use a mysql client end point in Ballerina to return the list of github ids.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4c6b5eafe9c9930e4eee3d80e20eb81f/href">https://medium.com/media/4c6b5eafe9c9930e4eee3d80e20eb81f/href</a></iframe><p><strong>2. Produce an array of Github Ids of employees who have not starred the repository</strong></p><p>We can obtain a Github access token by following <a href="https://help.github.com/articles/creating-a-personal-access-token-for-the-command-line/">https://help.github.com/articles/creating-a-personal-access-token-for-the-command-line/</a> . Then use that access token to create a Github client end point.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/1d83f7379d135125707f7d2c2b99c709/href">https://medium.com/media/1d83f7379d135125707f7d2c2b99c709/href</a></iframe><p>We can get a list of github user ids who have starred a repository by sending an http request. Following is a curl command which would generate a list.</p><pre>curl <a href="https://api.github.com/repos/MadhukaHarith92/ballerina-github-stars-tracking/stargazers?access_token=a8d66bb023876ad97f0e04dc916257583ec83f30">https://api.github.com/repos/MadhukaHarith92/ballerina-github-stars-tracking/stargazers?access_token=&lt;ACCESS_TOKEN&gt;</a></pre><p>Note that the above command would return only 30 users by default. We can increase this to a maximum of 100. If the star count of that repository is higher (in most cases) we have to iterate through each page and get them to an array which is shown as the following code.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/a1d746ae8ef53ba6e4353a81e84786b3/href">https://medium.com/media/a1d746ae8ef53ba6e4353a81e84786b3/href</a></iframe><p>So what we need to do now is figure out whether each employee has starred the repository or not. We can write a function to check if the employee’s github user id contains in the starred list we obtained earlier as follows.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6810e0496cd7d7d9091c8bd1cef539a0/href">https://medium.com/media/6810e0496cd7d7d9091c8bd1cef539a0/href</a></iframe><p><strong>3. Send email notification with the list of missing Github ids</strong></p><p>Next we will use the Gmail Api to send an email notification with a list of github repositories and details of employees who have not starred each repository. To do this we first need to obtain gmail oauth credentials by following this guidline: <a href="https://central.ballerina.io/wso2/gmail">https://central.ballerina.io/wso2/gmail</a> . We will then use the access token, refresh token, client id and client secret to create a gmail client endpoint.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/791bc5f04079425da1f663da98c491f6/href">https://medium.com/media/791bc5f04079425da1f663da98c491f6/href</a></iframe><p>Next we will create the email body by including the json array of repository and missing employee details. First lets create the email body. The email will contain one table with a summary of each repository (repo name, missing employee count )</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RtU6F7Gtqt5fdfwspDqMZw.png" /></figure><p>Upon clicking “SEND TO ALL” it will automatically generate an email addressing to all the missing employees requesting to star the repo as follows.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*61MGJQy5RRUIFUuv0FU7zQ.png" /></figure><p>The email notification will also contain tables per each repository with details of employees who have not starred that particular repo. By clicking on the email address it will automatically generate an email addressing to that particular employee requesting to star the repo as follows.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XKDEcgCz5A50bf3oR9COdw.png" /></figure><p>Here is the function to generate the email body using the json array of repository and employee details returned in getMissingIds function in step 2.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/9c3ada46f2d4dd081629fc750aa49a64/href">https://medium.com/media/9c3ada46f2d4dd081629fc750aa49a64/href</a></iframe><p>Now lets send the mail using the gmail client.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/9f2dc52a0e563e68e0545fe724ed6121/href">https://medium.com/media/9f2dc52a0e563e68e0545fe724ed6121/href</a></iframe><p><strong>4. Daily task to send the mail at 11.00 p.m</strong></p><p>We are not quite done yet. We need to automate this process so that an email notification would be sent to a mail list every day.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/8a65558c155872af6861c5c5da3a1adb/href">https://medium.com/media/8a65558c155872af6861c5c5da3a1adb/href</a></iframe><p><strong>5. Run the program</strong></p><p>Finally go to the terminal and run the command:</p><pre>ballerina run org &lt;COMMA SEPARATED GITHUB_REPOSITORY LIST&gt;</pre><p>You can find the complete program in <a href="https://github.com/MadhukaHarith92/automated-github-stars-tracker">https://github.com/MadhukaHarith92/automated-github-stars-tracker</a> . Oh and don’t forget to star the repo ;)</p><p>You can learn more about Ballerina in <a href="https://ballerina.io/learn/by-example/">https://ballerina.io/learn/by-example/</a> .</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7c705adb8441" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>