<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Ishadi Jayasinghe 👐 on Medium]]></title>
        <description><![CDATA[Stories by Ishadi Jayasinghe 👐 on Medium]]></description>
        <link>https://medium.com/@ishadij.12?source=rss-33f4e9a27c6d------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/2*CGApa_-vcpqR2ufBEPa9Nw.jpeg</url>
            <title>Stories by Ishadi Jayasinghe 👐 on Medium</title>
            <link>https://medium.com/@ishadij.12?source=rss-33f4e9a27c6d------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 21:20:13 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@ishadij.12/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Deploying your Springboot app with Kubernetes from scratch — Part 1]]></title>
            <link>https://medium.com/@ishadij.12/deploying-your-springboot-app-with-kubernetes-from-scratch-part-1-c6e16c0ec404?source=rss-33f4e9a27c6d------2</link>
            <guid isPermaLink="false">https://medium.com/p/c6e16c0ec404</guid>
            <category><![CDATA[port-forwarding]]></category>
            <category><![CDATA[java]]></category>
            <category><![CDATA[spring-boot]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[deployment]]></category>
            <dc:creator><![CDATA[Ishadi Jayasinghe ]]></dc:creator>
            <pubDate>Sun, 07 Jan 2024 17:22:41 GMT</pubDate>
            <atom:updated>2024-01-08T07:02:41.375Z</atom:updated>
            <content:encoded><![CDATA[<h3>Deploying your Springboot app with Kubernetes from scratch — Part 1</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/1*EaMuEQEfbChFnXOztxbucQ.png" /><figcaption>Source: <a href="https://kajalrawal.medium.com/kubernetes-spring-boot-4a5af9e5905c">https://kajalrawal.medium.com/kubernetes-spring-boot-4a5af9e5905c</a></figcaption></figure><p>Hi all, as the name suggests, this is the first part of a series of articles I’m going to take you for everything you need to know for deploying production ready springboot app via Kubernetes.</p><p>In this article, I’m covering the basic setup — simple springboot app with one GET api, deploying the app as a single pod, then via a replication-set, and finally via a deployment.</p><h4>Sample App</h4><p>Following are the build.gradle file, application.yaml, the controller, and the the dto class</p><pre>plugins {<br> id &#39;java&#39;<br> id &#39;org.springframework.boot&#39; version &#39;3.2.0&#39;<br> id &#39;io.spring.dependency-management&#39; version &#39;1.1.4&#39;<br>}<br><br>group = &#39;com.ishj&#39;<br>version = &#39;0.0.1-SNAPSHOT&#39;<br><br>java {<br> sourceCompatibility = &#39;17&#39;<br>}<br><br>repositories {<br> mavenCentral()<br>}<br><br>dependencies {<br> implementation &#39;org.springframework.boot:spring-boot-starter-web&#39;<br> testImplementation &#39;org.springframework.boot:spring-boot-starter-test&#39;<br>}<br><br>tasks.named(&#39;test&#39;) {<br> useJUnitPlatform()<br>}</pre><pre>server:<br>  servlet:<br>    context-path: /user-service<br>  port: 9090</pre><pre>package com.ishj.dcrpoc.controller;<br><br>import com.ishj.dcrpoc.dto.User;<br>import org.springframework.web.bind.annotation.GetMapping;<br>import org.springframework.web.bind.annotation.RestController;<br><br>@RestController<br>public class MainController {<br><br>    @GetMapping(&quot;/users/admin&quot;)<br>     public String getAdminName(){<br>        User user = new User(&quot;ishj&quot;, &quot;admin&quot;);<br>        return user.name();<br>    }<br>}</pre><pre>package com.ishj.dcrpoc.dto;<br><br>public record User (String name, String role) {<br>}</pre><p>Note that we have exposed our app via port 9090 in application.yaml above. Upon sucessful run, you should be able to get ‘ishj’ as the response when you execute</p><pre>curl http://localhost:9090/user-service/users/admin</pre><h4>Containerising the app</h4><p>First we have to create the docker file as we are planning to use docker runtime inside Kubernates<br>Doclerfile in the root folder level parallel to build folder</p><pre>FROM openjdk:17-oracle<br>WORKDIR /opt<br>ENV port 9090<br>EXPOSE 9090<br>copy build/libs/dcr-poc-0.0.1-SNAPSHOT.jar /opt/app.jar<br>ENTRYPOINT exec java $JAVA_OPTS -jar app.jar</pre><p>Here, we expose port 9090 from the container, copy our built jar to /opt folder, and then execute the jar using java -jar</p><p>Then we build the image<br><em>docker build -t user-service .</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ca24P3rfQAaOnpzP7PGpXg.png" /></figure><p>then we run the image we built<br><em>docker run — name user-service — rm -p 9091:9090 user-service</em></p><p>Here, we say to map outside port 9091 to inside port 9090 so that we can do<br><em>curl </em><a href="http://localhost:9091/user-service/users/admin"><em>http://localhost:9091/user-service/users/admin</em></a><em> </em>to invoke our api. We also can very well use <em>-p 9090:9090 </em>with no issues. I used 9091 to demonstrate how port mapping works here.</p><h3>Uploading the image to registry</h3><p>Now let’s upload our image to dockerhub (later we’ll cover how to use ECR for this). This is done so that later Kubernates can pull our image from Dockerhub. Steps are as follows.</p><ol><li>login to dockerhub ( need to create an account <a href="https://hub.docker.com/signup">here</a> ). Your username<br>for subsequent steps is your docker ID.<br><em>docker login</em></li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eOVKVsvprWCG3oOpQGEVig.png" /></figure><p>2. Now we need to tag our image in the form of &lt;username&gt;/image-name:tag ( tag is optional, but useful here. we can use tag to track versions of the image). After tagging, let’s verify whether our image is all good to push.<br><em>docker tag user-service ishjay/user-service:1.0.0</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5tSqYl-doGGQBiGHkbDpgw.png" /></figure><p>3. Pushing the image to dockerhub<br><em>docker push ishjay/user-service:1.0.0</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kUjgbWPnHXxJ5p9Slj3Zew.png" /></figure><p>4. Finally, let’s verify whether our image is exsiting in dockerhub via <a href="https://hub.docker.com/">portal</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pp1eQF3fkb4TiqFvJQGaDQ.png" /></figure><h3>Kubernetes Deployment</h3><p>Here, we’re using a local kubernates cluster we set up with <a href="https://minikube.sigs.k8s.io/docs/start/">minikube</a>. We also need <a href="https://kubernetes.io/docs/tasks/tools/">kubectl</a> utility which is the CLI we use to interact with k8s (kubernates) cluster. We can verify whether cluster is running using below.<br><em>kubectl cluster-info</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*s4hpjeMGx_EnzBs1RSUNvQ.png" /></figure><p>Now we’re all set. Let’s proceed for our deployments. Deployments to Kubernates are done through yaml files which contain resource definitions. We call these manifests also. A manifest mainly contain 4 core parts — apiVersion, kind, metadata, and spec.</p><h4>Deploying as a single pod</h4><p>pod-definition.yaml file</p><pre>apiVersion: v1<br>kind: Pod<br>metadata:<br>  name: user-service<br>  labels:<br>    tier: back-end<br>spec:<br>  containers:<br>    - name: user-service<br>      image: ishjay/user-service:1.0.0<br>      ports:<br>        - containerPort: 9090<br>      imagePullPolicy: Always</pre><p>Here, we have specified some labels, and then most importanly, name of our image at dockerhub ( note the username), port to be exposed. <br>We can now create our pod using<br><em>kubectl create -f pod-definition.yaml</em></p><p>now, if we see our pods using <em>kubectl get pods, we can see following, and then </em>we can also see our app logs using<em> kubectl logs user-service </em>(user-service — our pod name)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DWaR52VGHQv1MQ1oRRoLMQ.png" /></figure><p>Unfortunately, we cannot directly access the pod from outside because pods are inside a private network inside the cluster. However, with <em>port-forwarding</em>, a mechanism to map an external port of the cluster to the pod, we can still access the APIs exposed by the pod. <br>So, below, we’re forwading traffic we get under port 9092 to port 9090 of user-service pod. 9090 is the port through which our pod is accepting the traffic<br><em>kubectl port-forward user-service 9092:9090</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FEzOeKzT44V2ZjlGGLiYDA.png" /></figure><p>now we can access our api via <em>curl </em><a href="http://localhost:9092/user-service/users/admin"><em>http://localhost:9092/user-service/users/admin</em></a></p><h4>Deploying via a replica-set</h4><p>Above method of deploying as a pod is not appropriate as a solid deployment stratergy because there’s no fallback mechanism. For example, if the pods gets down for some reason ( peaks in requests, etc. ), our system will be down. Replicaset comes into play in this area as a solution providing a guarantee that the specified amount of pods will be running everytime in the cluster. <br>Let’s define the manifest</p><pre>apiVersion: apps/v1<br>kind: ReplicaSet<br>metadata:<br>  name: user-service-rs<br>  labels:<br>    app: backend<br>spec:<br>  template:<br>    metadata:<br>      labels:<br>        tier: back-end<br>    spec:<br>      containers:<br>        - name: user-service<br>          image: ishjay/user-service:1.0.0<br>  replicas: 2<br>  selector:<br>    matchLabels:<br>      tier: back-end</pre><p>Here, we can see we define 2 replicas for the pod. Means we’re assured that there are 2 pods running with the specified image everytime in the cluster. Let’s test by deleting one pod.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*foPbfhmi7dZjDtWFRZpG6g.png" /></figure><p>We can see a new pod has started running as soon as we deleted one pod.</p><h4>Deploying via a Deployment</h4><p>We saw how we can ensure we have our application is running everytime in the cluster. But what if we want to deploy a new version of our application? Deleting all pods and then deploying pods with our new image makes us loose the traffic we get during the downtime. Also, what if we want to rollback? Where to find our former state? This strategy, deploying as a deployment serves as a solution for above.</p><p>This primarily does two things :</p><ol><li>Recording the state of each deployment — this makes smooth rollbacks possible</li><li>Deployment strategy — one of the strategies ( also the default) here is ‘<em>Rolling Update’</em>. This one does deployments with no downtime.</li></ol><p>Let me explain a bit how rolling updates happen</p><p>Imagine we have two pods running. When we do a new deployment with this stratergy, first, 2 new pods are created and kept seperately (as a new replicaset). Then a single old pod will be taken down and a new pod will be made available for receiving traffic. When that’s successful, the remaining old pod will be taken down and the remaining new pod will be made available to receive traffic. This way, application experiences <strong>no downtime</strong>. Also, as additional bonus, kubernates will not take down original pods if the new pods don’t start properly.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*omh1JCv9_JH8cX9GpH-pZA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IPLMHybbFGnVvB3THOBJ2g.png" /></figure><p><em>Rolling Update deployment strategy</em></p><p>Let’s see the version of the image deployed in current pod. <br>We can see this by executing <em>kubectl describe pod &lt;pod-name&gt;</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LDrhrkgENVYwoBwTdQeMTw.png" /></figure><p>Then, let’s create another version of the image (2.0.0) and upgrade our deployment.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EkRDyGU8EECpuIvK0Ep97Q.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7YzCA9T5EFDp6i4nXyumtg.png" /></figure><p>We can update the definition file with the new verision and then apply the configurations again using <em>kubectl apply -f kube/app-deployment-definition.yaml</em></p><p>Let’s verify pod image now</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CQErOvEPeWaC6ZA57_MsmA.png" /></figure><p>We can see the image is updated two pods are running. Let’s run describe command on the deployment to see how deployment was done using <em>kubectl describe deployment user-service-deployment</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OYYYeikMNrSAzxnqG-Ruhg.png" /></figure><p>Interestingly, as expected, we can see how updates happened in the rolling-update strategy ( taking one old pod at a time)</p><p><em>Rollbacks</em></p><p>Now, let’s see how to do the rollback<br>Surprisingly, it’s as easy as running <br><em>kubectl rollout undo deployment &lt;deployment-name&gt; </em><br>Let’s try and see whether our image gets back to 1.0</p><p>Existing</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GmP6QeYPuBqQ0ALojMtUVQ.png" /></figure><p>Rollback</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BASzl8kL4P9cdd3U0zqh9A.png" /></figure><p>Verifying</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gunswI2zjpmW9WJIr9zyLA.png" /></figure><p>We can see the deployment is successfully now in previous state.</p><p>So, above we discussed briefly on the basics for deploying an app in kubernetes. In the next article, we will be discussing on networking so that we can access our deployment from outside. Stay tuned! :)</p><p>As usual, please free to comment any doubts/suggestions. Thanks!</p><p>References:<br><a href="https://learnk8s.io/spring-boot-kubernetes-guide">https://learnk8s.io/spring-boot-kubernetes-guide</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c6e16c0ec404" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Monitoring microservices through AWS Cloudwatch dashboards]]></title>
            <link>https://medium.com/@ishadij.12/monitoring-microservices-through-aws-cloudwatch-dashboards-7221c0cd5f1f?source=rss-33f4e9a27c6d------2</link>
            <guid isPermaLink="false">https://medium.com/p/7221c0cd5f1f</guid>
            <category><![CDATA[microservices]]></category>
            <category><![CDATA[monitoring]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[failure]]></category>
            <category><![CDATA[cloudwatch]]></category>
            <dc:creator><![CDATA[Ishadi Jayasinghe ]]></dc:creator>
            <pubDate>Mon, 06 Mar 2023 18:51:06 GMT</pubDate>
            <atom:updated>2023-04-04T12:01:51.633Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*sDGKZ_HOsRC8tpnb" /><figcaption>Image source: <a href="https://alpacked.io/blog/microservices-use-cases/">https://alpacked.io/blog/microservices-use-cases/</a></figcaption></figure><blockquote>Disclaimer — opinions are my own and content is not vetted/reviewed/approved by my employer.</blockquote><p>More often than not, especially with the advent of on-demand cloud services, we can see now most services in the form of microservices. Netflix, for example, has always been one of the key advocates for microservice-based architecture. They harness microservices to cater to their ever-increasing demand for their near-zero latency video streaming.</p><p>One downside of using a microservice-based architecture, as opposed to a monolith architecture, can be seen as the increased difficulty in monitoring. Adding to this, the emphasis on asynchronous communication between microservices makes this even more challenging since the failure reports would either be not received or received with a delay.</p><p>In this article, we’re going to see how we can effectively and rather easily use AWS <a href="https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_Dashboards.html">Cloudwatch Dashboards</a> for monitoring microservices. I’ll first quickly go through a simple use-case scenario of an online food delivery system and then let’s together view the status of our microservices through the Cloudwatch dashboard!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/411/1*HKMz0sLZCFFbn8OeYlQLcA.png" /></figure><p>The diagram above shows the interactions between the microservices in our food delivery system.</p><p>Order service -&gt; Identity service: to fetch user details<br>Order service -&gt; Payment service: to direct the user to an external payment gateway<br>Order service -&gt; Notification service: to send notifications to users on payment, order status<br>Order service -&gt; Delivery service: to connect to registered delivery partners<br>Order service -&gt; Restaurant service: to fetch restaurant details, to submit orders<br>Delivery service -&gt; Notification service: to send notifications on the delivery status<br>Delivery service -&gt; Payment service: to send notifications on the delivery status</p><p>Now that we’ve established a simple use case scenario, let’s finalize a set of errors services can receive. Here we’ll be ignoring errors that are application specific.</p><p>Timeout: Connection Timeout, Read Timeout, Socket Timeout, etc. <br>Failure: Different kinds of failures<br>Exception<br>Error<br>Service Unavailable</p><h3>Approach</h3><p>We will be utilizing Cloudwatch logs to detect errors our services might be experiencing. First, we will grep those errors using <a href="https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/MonitoringPolicyExamples.html">Metric Filters</a> and then we will configure <a href="https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/AlarmThatSendsEmail.html">Cloudwatch Alarms</a> to detect whether the errors are one-time occurrences (thus can be ignored) or it’s happening frequently (and need attention). Finally, we will set up a <a href="https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_Dashboards.html">Cloudwatch dashboard</a> to indicate the alarm status and service status ( in terms of alarm status) overall.</p><p>Let’s go! :)</p><p>Following are my log groups for each service.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nYqdv2jktxs0Pt8fXdljPQ.png" /></figure><p>Then I’m creating metric filters for each log group for the errors we’ve listed previously. Below is my metric filter for order-service. The filter pattern I’m using here is</p><pre>?&quot;Timeout&quot; ?&quot;timeout&quot; ?&quot;Failure&quot; ?&quot;Service Unavailable&quot; ?&quot;Error&quot;</pre><p>This filters out any log that’s containing either of the words within double quotes. You can customize this or even have multiple metric filters (have made a note at the end on the benefit of using multiple filters) based on your application error codes.</p><p>Further configs to my metric filter are as below. I’m creating a new namespace for my first metric filter and will be reusing the same namespace for subsequent filters of other services.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xISIKLiUFh5sgy6mQrTl5A.png" /></figure><p>After pushing some logs with simulated errors, we can see the metric dashboard like below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ff4KLQXj56n6CxCAMQRS6Q.png" /></figure><p>Now let’s configure Alarms for each of these filters. Below is my config for delivery service. This means that if there are 5 hits within 1 minute, we trigger the alarm. Feel free to play around other options to see what’s really matching with your application.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6GHGIAJDMhjgKvCeclVF1A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4_1qf2WxQW-X8uJmp-wnOw.png" /></figure><p>Below is the other well known advantage of using Cloudwatch alarms. We can configure SNS endpoints so that we can get notified when an alarm is breached. Below, I’m using my existing endpoint which is to trigger myself emails.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XsHinT2g95Ev_-vlZ7CW3g.png" /></figure><p>Likewise, I will be creating alarms for our other services.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qdv8DxKmKsrgHncdL4XXjA.png" /></figure><p>Now, final step. We’re creating our dashboards! Dashboard option (at the top) at the navigation panel let us create a dashboard with a name of our choosing. I named mine as service-stat</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XOnzR9_3-e583-adBkg9Ww.png" /></figure><p>Now, using the yellow + button at the right, you can add alarm status widgets to the dashboard for each of the alarms we created.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/230/0*qbeQgFun48lTOb1_.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*I8m400YPmtqeU5FL12uv4w.png" /></figure><p>That’s it! :) . Let me simulate some connection errors for payment service to see how our dashboard would show it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YOsXs0q50Ur6QYdX02at2A.png" /></figure><p>Additionally, we can also see we have an email notification for the same.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/234/0*kjO4lSM75ihct5T4.jpeg" /></figure><p>This is the end. Below, I’m discussing some points we can use to make more improvements.</p><p>Multiple metric filters instead of combining error codes to a single metric<br> — If we have a less number of services, it’ll be much easier to debug the issue when we have separate filters for each error. However, as the number of services increases, it will be hard to manage if we have too many filters and alarms configured in the system.</p><p>Aggregated alarms<br> — Ins addition to using one alarm for one metric filter as we did above, we can aggregate multiple alarms ( with logics such as ‘and’, ‘or’ ) and create a single alarm as well. One benefit of this is that instead of configuring to send us emails when the single metric filter is matched, we can configure to be notified if multiple filters are matched ( a significant issue). We can also configure multiple dashboards to view our service status on multiple levels this way. For example, one dashboard would show us a coarse view of our services ( using aggregated alarms) and another dashboard can be made to show us a more fine-tuned view of a single service.</p><p>So, here we come to the end. Please feel free to comment with any questions or suggestions. Thanks and have a nice day!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7221c0cd5f1f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Matrix Decomposition — Eigen decomposition and SVD]]></title>
            <link>https://medium.com/@ishadij.12/matrix-decomposition-eigen-decomposition-and-svd-1759d4ae08e7?source=rss-33f4e9a27c6d------2</link>
            <guid isPermaLink="false">https://medium.com/p/1759d4ae08e7</guid>
            <dc:creator><![CDATA[Ishadi Jayasinghe ]]></dc:creator>
            <pubDate>Wed, 29 Apr 2020 13:27:21 GMT</pubDate>
            <atom:updated>2020-04-29T13:46:21.995Z</atom:updated>
            <content:encoded><![CDATA[<h3>Matrix Decomposition — Eigen decomposition and SVD</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lFpSPfwHkhaJesIFq_SD5Q.png" /><figcaption>Image source: <a href="https://pixabay.com/images/id-5066967/">pixabay</a></figcaption></figure><p>Decomposition refers to factoring something into sub parts. So, we can think of Matrix decomposition as breaking a given matrix into sub matrices or vectors. Matrix decomposition is important as it provides insight into qualities and behavior of matrices. For example, due to the nature of the vectors resulting from eigen decomposition, eigen decomposition can be used in famous PCA (Principal Component Analysis) where we find the optimal compression mechanism for a given set of vectors. In this post, we’re going to look into two decomposition techniques; Eigen decomposition and Singular Value Decomposition.</p><h4>Eigen decomposition</h4><p>First, let’s be clear about eigen vectors and eigen values.</p><p>For square matrix <strong>A</strong> with the dimension as (<em>n</em> x <em>n</em>), if there’s a vector <strong><em>v</em></strong><em> </em>such that <strong>A<em>v</em></strong> = λ<strong><em>v</em></strong><em> </em>where λ is a scalar, we call <strong><em>v</em></strong> is an eigen vector of <strong>A</strong> and <em>λ</em> is the corresponding eigen value to <strong><em>v</em></strong>.</p><p>Okay, now let’s see what eigen decomposition is. Assume <strong>A</strong> has <em>n </em>orthonormal (vectors which has a unit norm while being linearly independent with each other) eigen vectors. We get these <em>n</em> vectors as columns and form a matrix <strong>V</strong>. Then we can get the eigen decomposition of <strong>A </strong>as,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/137/1*KyJIQgdhvVCzCoWzM8jsIA.png" /></figure><p>Here, diag(λ) refers to the diagonal matrix we form by initiating its diagonal with our eigen values.</p><p>Eigen vectors of a matrix are often interpreted as representing the volume of the matrix in space. For example, if we have any eigen value as zero, we interpret matrix has lost all its volume as it’s shrunk in the direction of the corresponding eigen vector. Also, the product of eigen values gives the determinant of the considered matrix. So if any of the eigen values is zero, the determinant is of the matrix is also zero. Eigen decomposition can also be used to optimize functions in the following form:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/130/1*GgsR012DRyqv0AITpQx0PQ.png" /></figure><p>where <strong><em>x</em></strong> is constrained to be a unit vector. Under this constraint, whenever <strong><em>x</em></strong> is an eigen vector of <strong>A</strong>, <em>f</em>(<strong><em>x</em></strong>) would take the corresponding eigen value. Also, the maximum value of <em>f</em>(<strong><em>x</em></strong>) would be given when <strong><em>x</em></strong> is the eigen vector of <strong>A</strong> corresponding to the maximum eigen value of <strong>A</strong>. Similarly, the eigen vector of <strong>A</strong> corresponding to the minimum eigen value would make <em>f</em>(<strong><em>x</em></strong>) reach its minimum. To summarize, when <strong><em>x</em></strong> is a unit vector, maximum of <em>f</em>(<strong><em>x</em></strong>) would be the maximum eigen value of <strong>A</strong> and the minimum would be the minimum eigen value. An interesting conclusion that can be derived from this is if all eigen values of <strong>A</strong> are positive (i.e. positive definite matrix), then <em>f</em>(<strong><em>x</em></strong>) will only be zero if and only if <strong><em>x</em></strong> is zero.</p><h4>Singular Value Decomposition</h4><p>Unlike Eigen decomposition, which can only be applied to square matrices, Singular Value Decomposition (SVD) can be applied to any real matrix. SVD on matrix <strong>A</strong> (<em>m</em> x <em>n</em>) will be in the form of</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/116/1*hniL4BtB547jn10ilQDYbg.png" /></figure><p>where <strong>U</strong> and <strong>V</strong> are square matrices in the dimensions of (<em>n</em> x <em>n</em>) and (<em>m</em> x <em>m</em>) respectively, and <strong>D</strong> is a diagonal matrix with the dimension (<em>m</em> x <em>n</em>). We call the values of <strong>D</strong> along the diagonal as singular values, columns of <strong>U</strong> as the left singular vectors, and columns of <strong>V</strong> as right singular vectors. It’s important to note that the left singular vectors are the eigen vectors of <strong>A</strong>*transpose(<strong>A</strong>) and similarly, right singular vectors are the eigen vectors of transpose(<strong>A</strong>)*<strong>A</strong>.</p><p>In a linear system of the form <strong>A<em>x</em></strong> = <strong><em>b</em></strong>, if the number of columns of <strong>A</strong> is greater than the number of rows, then multiple solutions can exist. On the other hand, if the number of rows are greater than the number of columns, then there’s a possibility for no solution to exist. As matrix inverse is defined only for square matrices, matrix inversion cannot be used in these scenarios where <strong>A</strong> is not a square matrix. For such instances, we can use Moore-Penrose Pseudoinverse which is a matrix introduced in the form of</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/139/1*TKJU8txlwHmEtrZyYgfpZg.png" /></figure><p>where we call <strong>A+</strong> as the pseudo inverse of <strong>A</strong>. Here, <strong>V</strong>, <strong>D</strong> and <strong>U</strong> are the matrices from SVD of <strong>A</strong>. We obtain <strong>D+</strong> by taking the reciprocal of the diagonal values of <strong>D</strong> and then transposing the matrix.</p><p>Unlike in matrix inversion, we can calculate the pseudo inverse for non-square matrices and use it to solve linear systems of the form <strong>A<em>x</em></strong> = <strong><em>b</em></strong> as shown below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/102/1*N0jO0r5dGfj4O6fos6X6GA.png" /></figure><p>However, as we discussed above, we can face the issues of having infinite solutions or no solution at all when trying to solve linear systems with non-square matrices. Technique above with the pseudo inverse handles the scenario of infinite solutions (greater column count) by producing the solution with the smallest euclidean norm (closest to the origin). It handles the scenario of no solution (greater row count) by producing a solution <strong><em>x</em></strong> such that euclidean norm of (<strong>A<em>x</em></strong>-<strong><em>b</em></strong>) is minimum.</p><p>So we discussed two key decomposition techniques available for matrices with the practical implications. I hope you enjoyed the post. Feel free to reach out to me for any confusions. Have a good day!</p><h4>Reference</h4><p><a href="http://www.deeplearningbook.org/slides/02_linear_algebra.pdf">Linear Algebra chapter</a> of <a href="http://www.deeplearningbook.org/front_matter.pdf">Deep Learning book</a> by MIT</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1759d4ae08e7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Behind word embeddings..]]></title>
            <link>https://medium.com/@ishadij.12/behind-word-embeddings-31352e716c2f?source=rss-33f4e9a27c6d------2</link>
            <guid isPermaLink="false">https://medium.com/p/31352e716c2f</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[naturallanguageprocessing]]></category>
            <dc:creator><![CDATA[Ishadi Jayasinghe ]]></dc:creator>
            <pubDate>Mon, 27 May 2019 22:58:55 GMT</pubDate>
            <atom:updated>2019-05-27T22:58:55.444Z</atom:updated>
            <content:encoded><![CDATA[<p><a href="https://medium.com/towards-data-science/in-depth-analysis/home"><strong>In-Depth Analysis</strong></a></p><h3>Behind word embeddings..</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/309/1*FPBk3YNgsaxooQQO8-6QDQ.png" /><figcaption>Image source : work by Mikolov et al.</figcaption></figure><p>Word embedding can casually be referred to the process of representing words through dense vectors. Basically, a word embedding built for a specific vocabulary is a matrix with the row count as the <em>vocab_size</em> and column count as the <em>embedding_size</em>. The embedding for a word with the id <em>i</em> in the vocabulary is the <em>iᵗʰ</em> row of the matrix. This <a href="https://towardsdatascience.com/introduction-to-word-embedding-and-word2vec-652d0c2060fa">blog</a> is awesome for a detailed insight on how to build them. In this article, we are going to discuss the meaning of the already built word embeddings.</p><p>Okay, here we go :)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/698/1*PxCrOdRp4OHu7B6torh0rw.png" /><figcaption>Figure 2 : Words represented by vectors close to the word embedding of the word ‘game’. Source : <a href="https://projector.tensorflow.org/">Embedding projector</a></figcaption></figure><p>The key most important characteristic of a word embedding matrix is the close proximity of the vectors of the words which share a similar context. Figure 2 exemplifies this scenario; the words shown on the image are the words related to the embedding vectors which are situated close to the word embedding of the word <em>‘game’.</em> Almost all the words there are seen pretty much in the context of the word <em>‘game’.</em></p><p>There is a much cooler meaning too. We can see that one cluster or similar words are connected to another cluster of similar words through vectors of equal ( roughly) lengths. This is explained in the first image here. There’s a vector with a certain length as the difference between the two word embeddings associated with the words <em>‘man’</em> and <em>‘woman’. </em>We can see that the difference between the two words <em>‘uncle’</em> and <em>‘aunt’</em> is more or less the same length. Same goes for the words <em>‘king’</em> and <em>‘queen’. </em>The following quote explains this;</p><blockquote>if we denote the vector for word i as xᵢ, and focus on the singular/plural relation, we observe that xₐₚₚₗₑ −xₐₚₚₗₑₛ ≈ x𝒸ₐᵣ −x𝒸ₐᵣₛ , x𝒻ₐₘᵢₗᵧ −x𝒻ₐₘᵢₗᵢₑₛ ≈ x𝒸ₐᵣ −x𝒸ₐᵣₛ , and<br>so on.[1]</blockquote><p>This observation is fascinating in that we can use this for language modelling as well. As for the above description, if we have to find the plural noun of the word ‘car’, we can simply get the embedding vectors of the plural word via,</p><blockquote>x𝒸ₐᵣₛ = x𝒸ₐᵣ + (xₐₚₚₗₑ −xₐₚₚₗₑₛ)</blockquote><p>More formally,</p><blockquote>to answer the analogy question a:b c:d where d is unknown, we find the embedding vectors xₐ, xᵦ , x𝒸 (all normalized to unit norm), and compute y = xᵦ − xₐ + x𝒸 . y is the continuous space representation of the word we expect to be the best answer.</blockquote><p>In reality, the differences would not be exactly similar and there might not be a word associated with the the new embedding vector <em>y </em>we got above. Therefore, we get the word of the embedding vector closest to the vector we got. Cosine similarity could easily be used here.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/309/1*MgRAlJNObZ_JcDmHd2yPcQ.png" /><figcaption>Finding the vector which is closest to the vector y via Cosine similarity.</figcaption></figure><p>So, to wrap up, we saw how we can find words using word embeddings. The embeddings are used for many, if not all, other text related tasks and have been proved to be helpful in elevating metric scores.</p><p>Thank you for reading! :)</p><p>References</p><p>[1] Mikolov, T., Yih, W.T. and Zweig, G., 2013. Linguistic regularities in continuous space word representations. In <em>Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies</em> (pp. 746–751).</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=31352e716c2f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[GANs are just kids when it comes to the discrete space ?!?]]></title>
            <link>https://medium.com/@ishadij.12/gans-are-just-kids-when-it-comes-to-the-discrete-space-9ead7f18f65a?source=rss-33f4e9a27c6d------2</link>
            <guid isPermaLink="false">https://medium.com/p/9ead7f18f65a</guid>
            <category><![CDATA[generative-adversarial]]></category>
            <category><![CDATA[naturallanguageprocessing]]></category>
            <category><![CDATA[text-generation]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[lstm]]></category>
            <dc:creator><![CDATA[Ishadi Jayasinghe ]]></dc:creator>
            <pubDate>Thu, 14 Feb 2019 10:36:46 GMT</pubDate>
            <atom:updated>2019-07-15T06:32:19.493Z</atom:updated>
            <content:encoded><![CDATA[<h3>GANs are just kids when it comes to the discrete space ?!?</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aW-J3SS104zWVqEy8vftIw.jpeg" /><figcaption>image source: <a href="https://www.metroweekly.com/wp-content/uploads/2015/04/maxresdefault.jpg">https://www.metroweekly.com/wp-content/uploads/2015/04/maxresdefault.jpg</a></figcaption></figure><p>When it comes to text generation models, we can see a couple of issues in recurrent models that use maximum likelihood approaches (MLE). One such commonly known issue is the exposure bias; during the predicting phase, these models have their previously generated words as their context which they might not have seen in the training phase. This gets severe when generating long sequences.</p><p>Generative Adversarial Nets(GANs) were proposed by Goodfellow et al.[1] basically for image generation. Here, two agents are trying to score against each other. One agent is the generator that tries to generate realistic images while the other agent, the discriminator tries to differentiate generated images from the real images. The training procedure for the generator is to maximize the probability of the discriminator making a mistake. GANs have recorded significant advances in image generation.</p><p>Yu et al.[2] extended GANs to text generation. This is bringing GANs to the discrete space. The discriminator of a GAN is considering only the final output from the generator to make the classification, but when it comes to sequence generation, we need to know the error for each word. Therefore the main challenge here is propagating the error to those individual word prediction phases. Various approaches which are reinforcement learning(RL) based and non-RL based have been suggested for this task.</p><p>When we analyze the evaluation metrics which have been used to compare the above approaches, some issues could be observed. If we focus on the measure used in SeqGan with its synthetic data, it scores a generated sentence based on the probability of it to be generated by the oracle. Oracle refers to a stationary LSTM model which is considered to have a certain data distribution p. If the Oracle assigns a high probability for a given sequence, that sequence can be thought as a sequence belonging to the distribution p. The issue with scoring sequences against an oracle is that it is completely blind to the diversity of the generated sequences i.e. if there’s a set of sequences with the same repeated sequence, it will score very high if that single sequence is in the distribution p. Zhu et al.[3] proposed a metric called self-BLEU which can be thought of accounting to diversity and Lu et al. proposed a metric called negative-BLEU which accounts for the quality. So we can think that if a model has a low self-BLEU (better diversity) and a high negative-BLEU (high quality), it is superior to a model that has the scores the other way. As simple this is, as per the literature, it’s impossible to compare models this way as no model simultaneously outperforms another w.r.t both metrics [4].</p><p>Temperature sweep[4] is a novel evaluation metric proposed to eliminate the above issues. This considers the entropy of models. (I think of entropy as the model’s ability to be adventurous. ;) ). A model with a high entropy would generate more diverse sentences whereas a model with a low entropy would stick to the sentence quality. Here, models are observed varying the temperature. High temperature leads to high entropy. The image below explains how to use this metric to evaluate and compare models.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/814/1*QpDVwVciPfOvzL-_zkSy1w.png" /><figcaption>Figure-1: Image source: work by Caccia et al.[4] Each figure plots inverse quality against inverse diversity. So in both axes, the lower, the better. The first image (left) shows the traditional way of comparing models. Under this setting, we can’t say which model is the best as the red model offers good quality while the blue model offers a good diversity. The second image (middle) shows the two models’ behavior with the temperature. So if the task at hand requires diverse data, we should go to the red model and the blue model if we need high-quality data. The third (right) image shows the curves of another two models. From here, we can say the blue model is better than the red model from every aspect.</figcaption></figure><p>Now the interesting part. :)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/527/1*Xpq17wHiy9WsiZr_MD3d9w.png" /><figcaption>Image source: work by Caccia et al[4]. This is related to the quality of generated sentences in the synthetic data experiment described above. Here α related to the temperature. As we can see, when we make α very low in MLE, we get it as the best model w.r.t. quality, but MLE just do this by producing the same sentence every time. So this is obviously not a good measure to evaluate models.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/437/1*voaS836TO5i06c0G2Lw3kg.png" /><figcaption>Image source: work by Caccia et al[4]. Despite the simplicity, we can see MLE outperforms official GANs like SeqGAN and LeakGAN. It more or less shows the same performance as CoT, but CoT employs more complex computations compared to MLE.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/445/1*24md_zK7LcVLwHm3jd2RWg.png" /><figcaption>Image source: work by Caccia et al[4]. This is the temperature sweep results for the task of generating sentences using EMNLP dataset. Here as well, we can see how MLE has best performed compared to GANs.</figcaption></figure><p>From above results, we can see how GANs are falling back behind traditional MLE approaches in the discrete space. While GANs are superior in the continuous space, it can be observed that there’s much work to do in extending them to the continuous space. Results above are based on the experiments done by Caccia et al[4]. Kudos to them :)</p><p>References<br>[1] Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A. and Bengio, Y., 2014. Generative adversarial nets. In <em>Advances in neural information processing systems</em> (pp. 2672–2680).</p><p>[2] Yu, L., Zhang, W., Wang, J. and Yu, Y., 2017, March. SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient. In <em>AAAI</em> (pp. 2852–2858).</p><p>[3] Zhu, Y., Lu, S., Zheng, L., Guo, J., Zhang, W., Wang, J. and Yu, Y., 2018. Texygen: A Benchmarking Platform for Text Generation Models. <em>arXiv preprint arXiv:1802.01886</em>.</p><p>[4] Caccia, M., Caccia, L., Fedus, W., Larochelle, H., Pineau, J. and Charlin, L., 2018. Language gans falling short. <em>arXiv preprint arXiv:1811.02549</em>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9ead7f18f65a" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>