<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Chamith Kodikara on Medium]]></title>
        <description><![CDATA[Stories by Chamith Kodikara on Medium]]></description>
        <link>https://medium.com/@ChamithKodikara?source=rss-2a4206fe48f6------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*hetM-XteAB9VHtt9LS0Osw.jpeg</url>
            <title>Stories by Chamith Kodikara on Medium</title>
            <link>https://medium.com/@ChamithKodikara?source=rss-2a4206fe48f6------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 09:17:05 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@ChamithKodikara/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Spring Batch Processing Example]]></title>
            <link>https://medium.com/@ChamithKodikara/spring-batch-processing-example-8472d4646f60?source=rss-2a4206fe48f6------2</link>
            <guid isPermaLink="false">https://medium.com/p/8472d4646f60</guid>
            <category><![CDATA[batch-processing]]></category>
            <category><![CDATA[java]]></category>
            <category><![CDATA[spring-boot-2]]></category>
            <category><![CDATA[spring-batch]]></category>
            <category><![CDATA[programming]]></category>
            <dc:creator><![CDATA[Chamith Kodikara]]></dc:creator>
            <pubDate>Mon, 02 Sep 2019 11:20:09 GMT</pubDate>
            <atom:updated>2019-09-02T11:20:09.953Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*X25YjzCl4a3eRrJoVIl5Wg.png" /></figure><p>Hi let’s have a quick look at <strong>Spring Batch Processing</strong>.</p><p><strong>First let’s see what batch processing is…</strong></p><p>Batch processing is a technique which process data in large groups (chunks) instead of single element of data. This is used to process high-volumes of data and do any modifications before processing with minimal human interaction. This has many advantages when dealing with large amount of data such as report generation, billing analysis systems and log analysis etc.</p><p><a href="https://www.bmc.com/blogs/what-is-batch-processing-batch-processing-explained/">What is Batch Processing? Batch Processing Explained</a></p><p>This is a basic idea of batch processing, And in this post we are going to take a look at the <strong>spring batch processing</strong></p><h4>Spring Batch Processing</h4><p>Spring batch is an opensource batch processing framework which is provided by spring.</p><blockquote>A lightweight, comprehensive batch framework designed to enable the development of robust batch applications vital for the daily operations of enterprise systems.</blockquote><p><a href="https://docs.spring.io/spring-batch/4.0.x/reference/html/spring-batch-intro.html#spring-batch-intro">Spring Batch Introduction</a></p><h4>How Batch processing works…..?</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/738/1*KYHNG9LqJK94MHqLQwCqog.png" /><figcaption><a href="https://docs.spring.io/spring-batch/trunk/reference/html/domain.html">https://docs.spring.io/spring-batch/trunk/reference/html/domain.html</a></figcaption></figure><p>Above is a basic structure of the spring batch. This is structured considering a normal batch processing architecture. Let’s see how each of these components works in spring batch.</p><p><strong><em>JobRepository</em></strong> — This manages the process or the condition of Job and Step. All the management data is stored to the spring batch tables in the database which are specified by spring batch</p><p><strong><em>JobLauncher</em></strong> — A simple interface for running a Job, and all possible ad-hoc executions. JobLauncher can be directly used by the user. But this won’t make any guarantee about whether its executed synchronously or asynchronously. It will depend on the implementation of the process.</p><p><strong><em>Job</em></strong> — Single execution unit that defines the series of how the process works. Job is an explicit abstraction which represents the configuration of a job specified by a developer.</p><p><strong><em>Step —</em></strong>Step is the processing unit of the Job, a Job can contain one or more steps depending on the logic we defines, Which we can define as chunk model or tasklet model. Same as a Job, Step is meant to explicitly represent the configuration of a step by a developer.</p><p><strong><em>ItemReader, ItemProcessor, ItemWriter</em></strong> — ItemReader and ItemWriter are the components that reads and writes data, convert data and files to Java objects and vice versa. And we can use ItemProcessor to process these data in between read and write, we can introduce any of the business logic and data conversions etc.</p><p>That was the basic background of how spring batch works and its main components. You can find more detailed one in spring batch documentation in below link.</p><p><a href="https://docs.spring.io/spring-batch/3.0.x/reference/html/spring-batch-intro.html#springBatchBackground">1. Spring Batch Introduction</a></p><h3>Getting Started….</h3><p>Now since we have a basic idea of what batch processing is, let’s build a sample application to see how spring batch processing works. For this, I’m going to use a spring boot application. To begin with let’s create a simple spring boot application using “SPRING INITIALIZR”</p><p><a href="https://start.spring.io/">Spring Initializr</a></p><p>Now we need to add spring batch features to our app. For this, we are going to add <strong><em>spring-boot-starter-batch </em></strong>to the pom.xml.</p><pre>&lt;dependency&gt;<br>    &lt;groupId&gt;org.springframework.boot&lt;/groupId&gt;<br>    &lt;artifactId&gt;spring-boot-starter-batch&lt;/artifactId&gt;<br>&lt;/dependency&gt;</pre><p>This will include the most compatible batch core and batch infrastructure version according to our spring boot version. Since I’m using the latest spring boot version (2.1.7.RELEASE) we are going to get <strong><em>batch 4.1.2.RELEASE</em></strong> to our application.</p><p>Now we have spring batch core and spring batch infrastructure in our application, we can start develop our sample batch application.</p><p>In this demo I’m going to get some sample data from a csv file and move those data in to MySQL db using spring batch. First, I’m going to create sample csv file call “orders.csv” with <strong><em>order_ref, amount, order_date, note </em></strong>fields<strong><em> </em></strong>in resources path and add few records.</p><p>Next I’m going to add DB connection properties to the application.properties.</p><pre># Platform configs<br>spring.application.name=batch-demo<br>spring.datasource.platform=org.hibernate.dialect.MySQL5Dialect<br>spring.datasource.url=jdbc:mysql://localhost:3306/batch_demo_db<br>spring.datasource.username=root<br>spring.datasource.password=admin</pre><pre>spring.jpa.show-sql=true</pre><p>Earlier I told you that spring batch is using DB table schema to keep management data for the spring batch process which are specified by spring batch. those tables are named with the prefix <strong><em>“BATCH_”</em></strong> in our database.</p><p>There are two ways to initialize these table first one is let spring batch to create these table automatically, for this we need to add this property in our application.properties file.</p><pre>spring.batch.initialize-schema=<em>always</em></pre><p>if we add “never” instead of “always” we can switch off the initialization.</p><p>Another way to initialize batch tables is create them manually, in this sample app I have added the schema script in the dbscript directory.</p><p>Now we have completed the background we need to start our application. We can start the implementation.</p><p>I’m going to create the entity class for orders to process these data to the DB. This is just a basic entity with id, name, amount, date and note fields.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/444/1*QbYCwAY2I__UKZEawdaxuA.png" /></figure><p>Let’s enable batch processing in our application. In order to do this we can use “@EnableBatchProcessing” annotation. This will enable spring batch features in our application. As a best practice you should add all the “@Enable” annotations in the main class of your application which will make your implementations easier. Since all your spring feature enabling will happens in one place, if you need to disable any feature you just need to remove correct annotation from the main class.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/488/1*u5uKmbvWSZhkI0mS4nf6gw.png" /></figure><p>Now let’s start our batch configuration. I’m creating a BatchConfig class in the app and autowire “JobBuilderFactory”, “StepBuilderFactory” in to it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/434/1*bFHh6WQ4WgwZ7FyxPxPY_w.png" /></figure><h4>JobBuilderFactory and StepBuilderFactory</h4><p>Spring batch use JobBuilderFactory to create the job builder, StepBuilderFactory to create step builder and initialize <strong><em>JobRepository</em> </strong>and <strong><em>PlatformTransactionManager</em> </strong>automatically.</p><h3>Config the ItemReader</h3><p>Now we need to configure the ItemReader to read data from order.csv file. I’m creating a spring bean using<strong><em> FlatFileItemReader&lt;T&gt; </em></strong>which is a child class of<strong><em> ItemReader </em></strong>Interface.</p><p><a href="https://docs.spring.io/spring-batch/trunk/apidocs/org/springframework/batch/item/file/FlatFileItemReader.html">FlatFileItemReader (Spring Batch 3.0.8.RLEASE API)</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/781/1*jgkq6UV74tOHK_HVu3G9tw.png" /></figure><p>In here I have created a FlatFileItemReaderBuilder for Order entity. I have given it a name and resource where our CSV file is and skip first line in order to skip CSV headers.</p><p>names identified the names of the fields that need to read, linemapper LineMapper interface custom implementation to map file line to the domain object.</p><h4><strong>LineMapper</strong></h4><p>We can implement custom line mapper to map lines from CSV file to domain object using LineMapper interface we can use <strong><em>FieldSetMapper</em></strong> implementation to map each field to domain object field and add it to the line mapper with custom delimiters.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/637/1*oIHdNqJyywEJWm1EhnXo4A.png" /></figure><h4>FieldSetMapper</h4><p>I have implement a custom OrderFieldMapper using FieldSetMapper to map each field to the domain Object.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/604/1*alqahjjBIOpLZdFMu_ePiQ.png" /></figure><h3>Config the ItemProcessor</h3><p>Now we have our ItemReader to read the data. We are now going to create ItemProcessor to process these data before we write them in to the database, for this I can create Custom ItemProcessor using <strong><em>ItemProcessor </em></strong>interface.</p><h4>OrderProcessor</h4><p>I have created custom OrderProcessor in order to convert order date from util.Date to time.LocalDateTime. This is just a simple processing I used in order to show how item processor works. You can use this to implement your custom logic that you need to run before you write the data in to database.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/807/1*JCVh_KD6SZ_H5mgRtxFTNA.png" /></figure><p>Now I’m going to define this as a spring bean in our BatchConfig configuration class.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/439/1*n0lUXqYKwI3-2UxEb5cNrA.png" /></figure><h3>Config the ItemWriter</h3><p>Now let’s config ItemWriter to write these processed data in to our MySQL DB. We can use either <strong><em>JdbcBatchWriter</em> </strong>to inset data using JDBC template or we can use <strong><em>JpaItemWriter </em></strong>which will use JPA and EntityManager to insert data in to the Database. For this example, I have used the JDBC one but you can use either of these which will be the best match for your requirement.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2sCKfFti15W2d9mrmCSYPA.png" /></figure><h3>Config the Job and Step</h3><p>Now since we have completed the reading processing writing implementations for our application we need to define a Step which will have those reader, processor, and writer using StepBuilderFactory to inject in to our batch Job.</p><h4>Step</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/618/1*sy21QraWUwFblqLx-fozfw.png" /></figure><p>In this I have set the reader, processor and writer in to the Step. And I have defined the chunk as 5, so this will read, process and write chunk of 5 data set for each transaction.</p><h4>Job</h4><p>Finally we need to config our Job.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/550/1*uCirkec_x13qV2yeSbigRw.png" /></figure><p>This is the Job Configuration, in this I have inject a step and a listener to the Job. We can implement custom listeners using <em>JobExecutionListenerSupport </em>to listen and log results before or after job execution.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/595/1*o3UNL5_AQY90TwA38lmRQQ.png" /></figure><p>These logs will print before and after the Job. We can have our custom implementation in these methods.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3JJbZY4vKbLa9B4kkIkNlQ.png" /></figure><p>This is the Basic Implementation example of the Spring Batch processing, for this one I have only used the chunk mode but they provide tasklet mode. Tasklets are meant to perform a single task within a step. You can have more idea about these two mode in here</p><p><a href="https://docs.spring.io/spring-batch/trunk/reference/html/configureStep.html">5. Configuring a Step</a></p><p>This is just a basic implementation that you can use to get a basic idea of how Spring Batch Works. After that you can do your own implementation to find best ways to use it for your requirement.</p><p>The <strong>full implementation </strong>and DB scripts of this can be found in github.</p><p><a href="https://github.com/ChamithKodikara/spring-batch-demo">ChamithKodikara/spring-batch-demo</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8472d4646f60" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Dockerizing Spring boot application]]></title>
            <link>https://medium.com/swlh/dockerizing-spring-boot-application-df5ae7dd1e37?source=rss-2a4206fe48f6------2</link>
            <guid isPermaLink="false">https://medium.com/p/df5ae7dd1e37</guid>
            <category><![CDATA[spring-boot]]></category>
            <category><![CDATA[containerization]]></category>
            <category><![CDATA[dockerfiles]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[docker-image]]></category>
            <dc:creator><![CDATA[Chamith Kodikara]]></dc:creator>
            <pubDate>Wed, 12 Jun 2019 05:29:40 GMT</pubDate>
            <atom:updated>2019-06-12T08:27:45.187Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*yBh1kqRSPjXKQa6NdkF8-A.png" /></figure><p>Hi I’m going to give a quick guide in <strong>Dockerizing Spring boot app.</strong></p><h4>First lets see what <strong>Dockerizing </strong>is…</h4><p>To understand this process of dockerizing we need to know about containers or <strong><em>application containerization</em></strong></p><h4><a href="https://searchitoperations.techtarget.com/definition/application-containerization-app-containerization">Application containerization (app containerization)</a></h4><blockquote>Application containerization is an OS-level virtualization method used to deploy and run distributed applications without launching an entire virtual machine (VM) for each app. Multiple isolated applications or services run on a single host and access the same OS kernel. Containers work on bare-metal systems, cloud instances and virtual machines, across Linux and select Windows and Mac OSes.</blockquote><p>Currently docker is the most common containerization technology. Containerize application using Docker is simply known as Dockerizing or Dockerization.</p><p><a href="https://www.docker.com/">Docker: Accelerated Container Application Development</a></p><blockquote><a href="https://docs.docker.com/get-started/">Docker is a platform for developers and sysadmins to <strong>develop, deploy, and run</strong> applications with containers. The use of Linux containers to deploy applications is called <em>containerization</em>. Containers are not new, but their use for easily deploying applications is</a></blockquote><p><a href="https://docs.docker.com/"></a></p><p>Above is a basic description about what docker is. Now lets create a sample application and dockerize it.</p><p>First we need to download and install docker in to the local machine. In order to do this we need to log into docker-hub (if you don’t have a dockerId create an account in docker-hub and log into it) then we can download and install docker in to our local machines.</p><p>And now we need to create sample application. I’m gonna create a sample spring-boot application with rest endpoint to test the app. we can generate sample project from <a href="https://start.spring.io/"><strong><em>“SPRING INITIALIZR”</em></strong></a><strong><em>.</em></strong></p><p><a href="https://start.spring.io/">Spring Initializr</a></p><p>Now we have a simple spring boot app. I’m gonna create a test endpoint in order to check when app is deployed in docker.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/602/1*iaNFPb6Ty4rGFybv8XZgJA.png" /><figcaption>host:port/test</figcaption></figure><p>Now lets start Dockerizing, first we need to add the Dockerfile to the app</p><h4><strong>Dockerfile</strong></h4><p>We know that docker runs docker images inside docker containers. Dockerfile is use to build these images, Dockerfile is a text file containing all the commands and instructions we need to call in the terminal when we are creating the docker images.</p><blockquote><a href="https://docs.docker.com/engine/reference/builder/">Docker can build images automatically by reading the instructions from a </a><a href="https://docs.docker.com/engine/reference/builder/">Dockerfile. A </a><a href="https://docs.docker.com/engine/reference/builder/">Dockerfile is a text document that contains all the commands a user could call on the command line to assemble an image. Using </a><a href="https://docs.docker.com/engine/reference/builder/">docker build users can create an automated build that executes several command-line instructions in succession.</a></blockquote><p>For this example I going to use basic docker file. You can get full understand what we can do with this Docker file from here</p><p><a href="https://docs.docker.com/develop/develop-images/dockerfile_best-practices/"></a></p><p>Now I have created a Dockerfile for the app,</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/876/1*OEbnaxDtss3-g_98docQcQ.png" /></figure><p><strong>FROM —T</strong>his instruct docker which base image is our image is based on and use that base image to build our image. Since I’m using java 8 I need my image to based on java 8 so I have added java 8 as the base image and docker will pull this from the public repository to my local machine to build the image. We can add more than one FROM in one Dockerfile and docker will create multiple images based on each base image.</p><p><strong>VOLUME— </strong>The VOLUME instruction creates a mount point with the specified name and marks it as holding externally mounted volumes from native host or other containers.</p><p><strong>ARG — </strong>The ARG instruction defines a variable that users can pass at build-time to the builder with the docker build.</p><p><strong>ADD — </strong>Adds a file or directory to the image.</p><p><strong>ENTRYPOINT — </strong>This tells docker how to start app after the image is started.</p><p><a href="https://docs.docker.com/engine/reference/builder/">Dockerfile reference</a></p><p>Now I can build the docker image using the following command</p><pre>docker build -t boot-docker-demo .</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*a6vBTwlTMrQEWp6GzxS0Qg.png" /><figcaption>The Docker image is created</figcaption></figure><p>Now I’m going to use maven to build the docker image using the docker file instead of <strong><em>docker build</em> </strong>for this we need to do few modifications to our pom.xml</p><p>First lets add <strong><em>docker-maven-plugin</em></strong> to the pom</p><h4><a href="https://github.com/spotify/docker-maven-plugin"><strong><em>docker-maven-plugin</em></strong></a></h4><blockquote>You can use this plugin to create a Docker image with artifacts built from your Maven project. For example, the build process for a Java service can output a Docker image that runs the service.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/770/1*X5ygRPBa2BrUq6qjIVCxzQ.png" /></figure><p>In here I have config how I need to create my docker image. Image name with prefix and jar file.</p><p>Now We can create docker image using maven</p><pre>mvnw install dockerfile:build</pre><p>By default the plugin will try to connect to docker on localhost:2375. Make sure you have exposed the docker demon on localhost:2375 or we can set the DOCKER_HOST environment variable to connect elsewhere.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WsAR0fmtvvVmxivsQkg0YQ.png" /></figure><p>If you haven’t exposed the docker demon it will throw this error</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*a_B8866QGNovai5aM0Q4SQ.png" /></figure><p>Once we have exposed this and Dockerfile and pom file are updated once we run the <strong><em>mvnw install dockerfile:build </em></strong>docker will create the docker image</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/989/1*ooRa1F-No9zbKtamOv8XYg.png" /></figure><p>When build is running for the first time if we don’t have the base images in our local machine the build will take time to pull all the images.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/891/1*gb01iLAWoOlHAlT4udNIKg.png" /></figure><p>Once we have base images in local machine the docker image will create in matter of seconds.</p><p>And now if I list all the docker images using <strong><em>docker images </em></strong>command we can see our base image and our images</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fRklKmD9KMnzw-kclhdPzA.png" /></figure><p>To run our docker image we can use below command</p><pre>docker run -p 8080:8080 -t your_docker_image_name</pre><p>This will create new container and start docker image and run application on a Linux environment.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cKDJocKGclZ84DISnUy22Q.png" /></figure><p>We can see our application running on a separate container.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*g70ntp5nHovIuI4yfUAfgA.png" /></figure><p>To Run your docker images on spring profiles is as easy as passing an environment to the docker run command.</p><pre>docker run -e &quot;SPRING_PROFILES_ACTIVE=prod&quot; -p 8080:8080 -t image_name</pre><p>As a plus I’m gonna show you guys a tool that we can use to manage our docker environments. This is called portainer</p><p><a href="https://www.portainer.io/">Portainer - One platform, not twelve tools - Portainer</a></p><p>We can install portainer to our local machine using following command</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*f1i5H920OeVGvShk9B5opw.png" /></figure><p><a href="https://www.portainer.io/installation/">Installation - Auckland, Singapore, San Francisco | Portainer.io</a></p><p>Once we start this portainer will be running on the given port on localhost</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IRc0NpksSm7SeMwbmcyGmA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ciV2zb3nyOG7Vup5nU47-w.png" /></figure><p>This can be used to easily manage our docker images, containers. Read more about this on <a href="https://www.portainer.io/">here</a></p><p>And for the <strong>full implementation</strong> of dockerizing spring boot applications can be found in github</p><p><a href="https://github.com/ChamithKodikara/boot-docker-demo">ChamithKodikara/boot-docker-demo</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=df5ae7dd1e37" width="1" height="1" alt=""><hr><p><a href="https://medium.com/swlh/dockerizing-spring-boot-application-df5ae7dd1e37">Dockerizing Spring boot application</a> was originally published in <a href="https://medium.com/swlh">The Startup</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[API Documentation & Application Monitoring in Spring Boot with Swagger, Prometheus & Grafana]]></title>
            <link>https://medium.com/@ChamithKodikara/api-documentation-application-monitoring-in-spring-boot-with-swagger-prometheus-grafana-af7f7bec0e1a?source=rss-2a4206fe48f6------2</link>
            <guid isPermaLink="false">https://medium.com/p/af7f7bec0e1a</guid>
            <category><![CDATA[actuator]]></category>
            <category><![CDATA[spring-boot]]></category>
            <category><![CDATA[swagger]]></category>
            <category><![CDATA[grafana]]></category>
            <category><![CDATA[prometheus]]></category>
            <dc:creator><![CDATA[Chamith Kodikara]]></dc:creator>
            <pubDate>Thu, 11 Apr 2019 10:55:04 GMT</pubDate>
            <atom:updated>2019-04-11T10:55:04.374Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*ExdOzEhJFQEgfjBcuT47gw.png" /></figure><p>Hi I’m gonna demonstrate a basic example of how to document APIs in spring boot application using <a href="https://swagger.io/">swagger</a>, produce application metrics using spring boot Actuator and <a href="http://micrometer.io/">Micrometer</a>, collect metrics using <a href="https://prometheus.io/">Prometheus</a>, visualize and monitor them using <a href="https://grafana.com/">Grafana</a>. Lets DIVE IN……</p><h3>What is API Documentation?</h3><blockquote>API documentation is a technical guide which contains how to use, integrate each API in a System. Which contains all the information required to use the API like details about the function of individual API (API user manual), classes, return types and more with tutorials and examples.</blockquote><blockquote>Previously these documentations were managed by creating manual documentations using text editors and other documenting tools, keep documents up-to date with the rapid evolution of your API’s functionality is hard to do with manual documentations.</blockquote><blockquote>API description formats like the <a href="http://swagger.io/specification/">OpenAPI/Swagger Specification</a> have automated the documentation process, making it easier for teams to generate and maintain them.</blockquote><p><a href="https://swagger.io/blog/api-documentation/what-is-api-documentation-and-why-it-matters/">Why Does API Documentation Matter? | Swagger Blog</a></p><h3><strong>Swagger</strong></h3><blockquote>Swagger open source and pro tools have helped millions of API developers, teams, and organizations deliver great APIs.Swagger takes the manual work out of API documentation, with a range of solutions for generating, visualizing, and maintaining API docs.</blockquote><p>Now lets take a quick look at how to create a sample API documentation with Swagger.</p><p>As usual, I’m gonna use <a href="https://start.spring.io/"><strong>Spring Initializr</strong></a><strong> </strong>to create a sample maven project.</p><p>And we can add swagger dependencies to pom.xml</p><pre>&lt;!--SpringFox swagger dependencies for API documentation--&gt;<br>&lt;dependency&gt;<br>    &lt;groupId&gt;io.springfox&lt;/groupId&gt;<br>    &lt;artifactId&gt;springfox-swagger2&lt;/artifactId&gt;<br>    &lt;version&gt;${springfox-version}&lt;/version&gt;<br>&lt;/dependency&gt;</pre><p>After these dependencies are added we need to enable swagger in our app in order to provide the automated documentation. For that we need to use @EnableSwagger2As a best-practice we should always use these @Enable annotations in the main class of the App.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/667/1*1T-jNz4AgYqpANki3faH7A.png" /><figcaption>@EnableSwagger2, @EnableJpaAuditing are added in main class.</figcaption></figure><p>The @EnableSwagger2 annotation enables Swagger 2 support in the application by registering all required beans in application context.</p><p>Now we need to configure the Springfox <a href="http://springfox.github.io/springfox/javadoc/2.7.0/springfox/documentation/spring/web/plugins/Docket.html">Docket</a>. This provides the primary API configuration with sensible defaults and convenient methods for configuration. In this I have created the Docket Bean in my AppConfig class.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/673/1*CBt8DY2uOoAeU-uE179pgQ.png" /></figure><p>In this Docket bean <em>select()</em> method returns an instance of <em>ApiSelectorBuilder</em>, which we can use to control what endpoints we need to document. In here I have added a basic config in order to use any endpoint in the App.</p><p>Now that we are done with our basic swagger configuration, we can move in to the actual documentation part in our REST APIs.</p><p>To show how to do this I have created a sample Rest Controller called <em>CustomerController. </em>I’m going to use this controller to show how to create a basic documentation.</p><p>For this we need few annotations which are provided by swagger</p><blockquote><strong>@Api</strong></blockquote><blockquote>In Swagger 2.0, resources were replaced by tags, and this impacts the @Api annotation. It is no longer used to declare a resource, and it is now used to apply definitions for all the operations defined under it.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/662/1*VBHL-bZ3IrQzeQL-T_VTwA.png" /></figure><blockquote><strong>@ApiOperation</strong></blockquote><blockquote>The @ApiOperation is used to declare a single operation. An operation is considered a unique combination of a path and a HTTP method.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/912/1*qfh-E2SBMYX88gI7T5-sWg.png" /></figure><p>In the customer create API I have used a few extra values inside <em>@ApiOperation</em></p><blockquote>The value of the annotation is a short description of the API. Since this is displayed in the list of operations in Swagger-UI and the location is limited in size, this should be kept short (preferably shorter than 120 characters). The notes allows you to give significantly more details about the operations. response is the return type of the method.</blockquote><p>You can get the full Documentation for these annotations in the below link</p><p><a href="https://github.com/swagger-api/swagger-core/wiki/Annotations-1.5.X">swagger-api/swagger-core</a></p><p>Now we have the sample API with documentation But we need a way to show this in a sensible way. For this, swagger provides us with Swagger-UI</p><p>Swagger UI is a built in user interface which makes the UX of the documentation way better. We can add this in to our app by simply adding the below dependency in our pom.xml</p><pre>&lt;dependency&gt;<br>    &lt;groupId&gt;io.springfox&lt;/groupId&gt;<br>    &lt;artifactId&gt;springfox-swagger-ui&lt;/artifactId&gt;<br>    &lt;version&gt;${springfox-version}&lt;/version&gt;<br>&lt;/dependency&gt;</pre><p>Now we can view the documentation using this url <a href="http://localhost:8080/swagger-ui.html"><em>http://localhost:8080/swagger-ui.html</em></a><em> </em>and it looks like this</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LVQ6SWSCao_6w6QtDWKHsw.png" /><figcaption>Home</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/872/1*ILrXUrx0LVdsT0dkPa454g.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/980/1*L5-XbM9mbBjJc0zFJRYrog.png" /></figure><p>To the left, you can see how my Customer controller documentation is listed.</p><p>To the left, we can see all the details of our API, and we can also test the API using <em>Try it out</em> button.</p><p>So as you can see this is a better way to provide API documentation in your Application. We can customize this as we want and it will automatically generate updated Documentation each time we update our actual API.</p><p>Now lets take a quick look into our next topic, Application Monitoring</p><h3><strong>Application Monitoring</strong></h3><blockquote>What is Application Monitoring ?</blockquote><blockquote>Application monitoring is a process that ensures that a software application processes and performs in an expected manner and scope. This technique routinely identifies, measures and evaluates the performance of an application and provides the means to isolate and rectify any abnormalities or shortcomings.</blockquote><blockquote>Application monitoring is also known as application performance monitoring (APM) and application performance management (APM).</blockquote><blockquote>— <a href="https://www.techopedia.com/definition/29133/application-monitoring">www.techopedia.com</a> —</blockquote><p>In this I’m gonna use Spring Boot 2 application with Micrometer, Prometheus and Grafana.</p><p>Now I’m gonna add actuator in to my App.</p><pre>&lt;dependency&gt;<br>    &lt;groupId&gt;org.springframework.boot&lt;/groupId&gt;<br>    &lt;artifactId&gt;spring-boot-starter-actuator&lt;/artifactId&gt;<br>&lt;/dependency&gt;</pre><p>This is gonna activate actuator endpoints like <em>info, loggers, health and metrics</em> in our application. Additionally, we can create custom actuator endpoints too.</p><p><a href="https://docs.spring.io/spring-boot/docs/current/reference/html/production-ready-endpoints.html">53. Endpoints</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/682/1*TzC8jfLVmJpFJUL20hrfTA.png" /></figure><p>Actuator exposes few endpoints to monitor our application, one of them is metrics endpoint, this endpoint is used to expose system and application level metrics via HTTP. To enable these endpoints we need to add below property in the <em>application.properties</em> file.</p><p>We can use <strong>“<em>management.endpoints.web.exposure.include=metrics” </em></strong>to just expose metrics or<strong> “<em>management.endpoints.web.exposure.include=*” </em></strong>to expose all. After this we can get metrics by<em> </em><a href="http://localhost:8080/actuator/metrics"><em>http://localhost:8080/actuator/metrics</em></a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/304/1*_XZtiRDOAeXi_cFWxRNqXw.png" /><figcaption>Some of the metrics</figcaption></figure><p>We can also provide metrics using <a href="http://micrometer.io/"><strong>Micrometer</strong></a><strong> </strong>in spring. These metrics will continuously change and keep in memory by the MeterRegistry. And we can use <a href="https://prometheus.io/"><strong>Prometheus </strong></a>with micrometer.</p><p>Prometheus is an open source tool use to monitoring metrics. It works in a pull-based manner, makes HTTP requests to our metric endpoint with time intervals and store these metrics in its own time-series database. Prometheus also provides Graphs to make queries over these metrics with its own query language PromQL. It provides basic graphics to visualize metrics. Prometheus also has an alert plugin to produce alerts according to metrics values.</p><p>For this we need to add micrometer and prometheus in to our App.</p><pre>&lt;!-- Micormeter dependencies for track app metrics --&gt;<br><br>    &lt;dependency&gt;<br>        &lt;groupId&gt;io.micrometer&lt;/groupId&gt;<br>        &lt;artifactId&gt;micrometer-core&lt;/artifactId&gt;<br>    &lt;/dependency&gt;<br>    &lt;dependency&gt;<br>        &lt;groupId&gt;io.micrometer&lt;/groupId&gt;<br>        &lt;artifactId&gt;micrometer-registry-prometheus&lt;/artifactId&gt;<br>    &lt;/dependency&gt;<br>&lt;/dependencies&gt;</pre><p>and expose prometheus endpoint using <em>application.properties </em>we can use <strong>“<em>management.endpoints.web.exposure.include=metrics,prometheus”</em></strong><em> </em>to just expose metrics and prometheus or <strong>“<em>management.endpoints.web.exposure.include=*” </em></strong>to expose all. After this we can get metrics by<em> </em><a href="http://localhost:8080/actuator/prometheus"><em>http://localhost:8080/actuator/prometheus</em></a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*fNS8TO2UFCBfcVFLj6AQ-g.png" /><figcaption>Some of the metrics provides by micrometer and prometheus</figcaption></figure><p>Now my metrics are ready to scrape by Prometheus, I have to create prometheus.yml to add prometheus configurations, I’m gonna add just basic configs for this example</p><pre><strong>scrape_configs</strong>:<br>  - <strong>job_name</strong>: spring-boot-monitoring-demo<br>    <strong>metrics_path</strong>: /actuator/prometheus<br>    <strong>static_configs</strong>:<br>      - <strong>targets</strong>: [&#39;192.168.1.154:8080&#39;,&#39;192.168.1.154:8180&#39;]</pre><p>Now prometheus is going to scrape data from these two targets.</p><p>I’m gonna use docker installation to install and start prometheus on my local machine. for this I’m gonna use below command</p><pre>docker run -d -p 9090:9090 -v location-of-my/prometheus.yml:/etc/prometheus/prometheus.yml prom/prometheus</pre><p>Now I can access prometheus dashboard in <a href="http://localhost:9090/targets"><em>http://localhost:9090/targets</em></a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MHaApwPwbs6Y9ixad3hX4w.png" /><figcaption>Prometheus targts</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*D48KLUwBtfQ2rYlHl_iVQQ.png" /></figure><p>And as I said before prometheus provides basic dashboard graph options</p><p>But If we need high-end nice looking dashboard to visualize our metrics we can use <a href="https://grafana.com/"><strong>Grafana</strong></a><strong>. </strong>Grafana gets metrics from our metric database and allows us to query, visualize and alert depending on our metrics. It supports many metric databases such as Prometheus, Influxdb, Graphite, Mysql etc</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kxD0RhmgY0qgIltjYiZTKg.png" /></figure><p>To add Grafana dashboard for our App we have to install and config the Grafana, To install Grafana I’m gonna use docker again</p><pre>docker run -d -p 3000:3000 grafana/grafana</pre><p>We can log into Grafana <a href="http://localhost:3000/"><em>http://localhost:3000</em></a> with default username and password as admin.</p><p>After logging in to the dashboard we need to config the data source, and we need to create prometheus data source in here.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/685/1*9V9Kleg5B3X-jta6W8VTpw.png" /></figure><p>we need to provide our prometheus url and create the datasource.</p><p>Grafana has a pack of dashboards we can use for our application and for this example I’m gonna use<strong> </strong><a href="https://grafana.com/dashboards/4701"><strong><em>JVM (Micrometer) by mweirauch</em></strong></a><strong><em> </em></strong>for our application.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FU_PDoi6iI_Zy-n9PN8j6g.png" /></figure><p>We can add this by adding dashboard number in the left UI. (For this dashboard its 4701)</p><p>and after we import this dashboard it will give us an amazing dashboard like this</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*X-AlleJkf6PIgKNj50ddTQ.png" /></figure><p>So this is a basic walkthrough of how to implement Application monitoring for spring boot 2 with Micrometer, Prometheus and Grafana.</p><p>You can use this to get basic idea and create your own awesome implementation with Documentation for your APIs and Dashboards to monitor the performance of your Application.</p><p>The <strong>full implementation</strong> of this can be found in github</p><p><a href="https://github.com/ChamithKodikara/monitoring-demo">ChamithKodikara/monitoring-demo</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=af7f7bec0e1a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[File Upload Download with Spring boot and Amazon S3]]></title>
            <link>https://medium.com/@ChamithKodikara/file-upload-download-with-spring-boot-and-amazon-s3-24c94b85c600?source=rss-2a4206fe48f6------2</link>
            <guid isPermaLink="false">https://medium.com/p/24c94b85c600</guid>
            <category><![CDATA[springcloudaws]]></category>
            <category><![CDATA[s3]]></category>
            <category><![CDATA[spring-boot]]></category>
            <category><![CDATA[file-management]]></category>
            <category><![CDATA[aws]]></category>
            <dc:creator><![CDATA[Chamith Kodikara]]></dc:creator>
            <pubDate>Mon, 21 Jan 2019 11:44:07 GMT</pubDate>
            <atom:updated>2019-01-21T11:44:07.897Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*r31z3lqlp8tGffdv3xzzgQ.jpeg" /></figure><p>Hey Guys, Lets have a quick look on how to handle a file storage(Digital archiving) of Document Management(Doc-Man) system using Spring boot and Amazon S3.</p><blockquote>“Digital archiving” — The long-term archiving of digital media</blockquote><p>In this I’m gonna show how to manage file upload, download, delete functions using sample app with Spring boot and Amazon S3.</p><blockquote><strong>What Is Amazon S3?</strong></blockquote><blockquote>Amazon Simple Storage Service is storage for the Internet. It is designed to make web-scale computing easier for developers.</blockquote><blockquote>Amazon S3 has a simple web services interface that you can use to store and retrieve any amount of data, at any time, from anywhere on the web. It gives any developer access to the same highly scalable, reliable, fast, inexpensive data storage infrastructure that Amazon uses to run its own global network of web sites. The service aims to maximize benefits of scale and to pass those benefits on to developers.</blockquote><blockquote>This guide explains the core concepts of Amazon S3, such as buckets and objects, and how to work with these resources using the Amazon S3 application programming interface (API). — aws.amazon.com</blockquote><p><strong>Amazon Simple Storage Service (Amazon S3)</strong></p><blockquote>Amazon Simple Storage Service (Amazon S3) is an object storage service that offers industry-leading scalability, data availability, security, and performance. This means customers of all sizes and industries can use it to store and protect any amount of data for a range of use cases, such as websites, mobile applications, backup and restore, archive, enterprise applications, IoT devices, and big data analytics. Amazon S3 provides easy-to-use management features so you can organize your data and configure finely-tuned access controls to meet your specific business, organizational, and compliance requirements. Amazon S3 is designed for 99.999999999% (11 9’s) of durability, and stores data for millions of applications for companies all around the world. — <a href="https://aws.amazon.com/">aws.amazon.com</a></blockquote><p><a href="https://aws.amazon.com/s3/">Amazon S3 - Cloud Object Storage - AWS</a></p><p>I’m gonna create a sample spring boot application to show how this can be done in a simple way. I have used the <a href="https://start.spring.io/"><strong><em>“SPRING INITIALIZR”</em></strong></a> to initialize a sample maven project.</p><p>Now we can add the AWS sdk to our application, to this we can either use “aws-java-sdk” from maven</p><pre>&lt;!-- https://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk --&gt;<br>&lt;dependency&gt;<br>    &lt;groupId&gt;com.amazonaws&lt;/groupId&gt;<br>    &lt;artifactId&gt;aws-java-sdk&lt;/artifactId&gt;<br>    &lt;version&gt;1.11.486&lt;/version&gt;<br>&lt;/dependency&gt;</pre><p>or we could use spring-cloud-starter-aws.</p><pre>&lt;dependency&gt;<br>    &lt;groupId&gt;org.springframework.cloud&lt;/groupId&gt;<br>    &lt;artifactId&gt;spring-cloud-starter-aws&lt;/artifactId&gt;<br>&lt;/dependency&gt;</pre><p>since I’m using spring cloud I have used the “spring-cloud-starter-aws” for this sample app.</p><p>Now we have successfully setup a basic app for our demo. Now we need to add user credentials of “AWS iam” user to create programmatically access to the AWS services.</p><h4>AWS Identity and Access Management (IAM)</h4><blockquote>AWS Identity and Access Management (IAM) enables you to manage access to AWS services and resources securely. Using IAM, you can create and manage AWS users and groups, and use permissions to allow and deny their access to AWS resources.</blockquote><blockquote>IAM is a feature of your AWS account offered at no additional charge. You will be charged only for use of other AWS services by your users.</blockquote><blockquote>To get started using IAM, or if you have already registered with AWS,</blockquote><p><a href="https://aws.amazon.com/iam/?sc_channel=PS&amp;sc_campaign=acquisition_IN&amp;sc_publisher=google&amp;sc_medium=ACQ-P%7CPS-GO%7CBrand%7CDesktop%7CSU%7CSecurity%7CIAM%7CIN%7CEN%7CText&amp;sc_content=aws_iam_p&amp;sc_detail=aws%20iam&amp;sc_category=Security&amp;sc_segment=293636784626&amp;sc_matchtype=p&amp;sc_country=IN&amp;s_kwcid=AL!4422!3!293636784626!p!!g!!aws%20iam&amp;ef_id=Cj0KCQiA-JXiBRCpARIsAGqF8wWgdmsFjafVpoQcFIxky6YmXK2qwk8RiY2iiRwOau1-llDrwOG7UPEaAuAmEALw_wcB:G:s">AWS Identity &amp; Access Management</a></p><p>And we also can add the bucket name in property file too</p><pre>#AWS properties<br>cloud.aws.stack.auto=false<br>cloud.aws.region.static=your AWS account region<br>cloud.aws.credentials.accessKey=AWS IAM user access key<br>cloud.aws.credentials.secretKey=AWS IAM user secret key<br><br>app.awsServices.bucketName=my-demo-s3-test</pre><p>After creating the S3 bucket we can add customized policy to the bucket using bucket policy or IAM settings.</p><p>After permissions are set for the bucket we can use user credentials (“accessKey” and “secretKey”) to access S3 bucket. To this we can create Amazon S3 Client using “AmazonS3” in aws-sdk</p><p><strong>Creating the Spring Bean</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/633/1*SzN-28Vj2e-Q0P9mqosMIQ.png" /></figure><p>in this @Value will get the property from application.properties file “cloud.aws.region.static” property.</p><p>Since i have used “cloud.aws.region.static=your AWS account region” it will show like this in the code.</p><p>Now we can use the “amazonS3Client” to access S3 bucket we can upload download delete in S3 bucket</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/637/1*G_RZ_Q7o8ieRA2IDI4NChQ.png" /></figure><p>in S3 bucket objects are stored as key value pairs so if we need to create a folder inside the bucket we can add folder-name/fileName in fileName param. and in AWS S3 bucket dashboard we can see folder inside the bucket.</p><blockquote><strong>Objects</strong></blockquote><blockquote>Objects are the fundamental entities stored in Amazon S3. Objects consist of object data and metadata. The data portion is opaque to Amazon S3. The metadata is a set of name-value pairs that describe the object. These include some default metadata, such as the date last modified, and standard HTTP metadata, such as Content-Type. You can also specify custom metadata at the time the object is stored.</blockquote><blockquote>An object is uniquely identified within a bucket by a key (name) and a version ID. For more information.</blockquote><blockquote><strong>Keys</strong></blockquote><blockquote>A key is the unique identifier for an object within a bucket. Every object in a bucket has exactly one key. Because the combination of a bucket, key, and version ID uniquely identify each object, Amazon S3 can be thought of as a basic data map between “bucket + key + version” and the object itself. Every object in Amazon S3 can be uniquely addressed through the combination of the web service endpoint, bucket name, key, and optionally, a version. For example, in the URL http://doc.s3.amazonaws.com/2006–03–01/AmazonS3.wsdl, “doc” is the name of the bucket and “2006–03–01/AmazonS3.wsdl” is the key.</blockquote><blockquote>For more information about object keys</blockquote><p>The <strong>full implementation </strong>and DB scripts of this can be found in github</p><p><a href="https://github.com/ChamithKodikara/s3-demo">ChamithKodikara/s3-demo</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=24c94b85c600" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Spring Boot 2, Quartz 2 Scheduler Integration]]></title>
            <link>https://medium.com/@ChamithKodikara/spring-boot-2-quartz-2-scheduler-integration-a8eaaf850805?source=rss-2a4206fe48f6------2</link>
            <guid isPermaLink="false">https://medium.com/p/a8eaaf850805</guid>
            <category><![CDATA[scheduler]]></category>
            <category><![CDATA[java]]></category>
            <category><![CDATA[spring-boot]]></category>
            <category><![CDATA[quartz]]></category>
            <dc:creator><![CDATA[Chamith Kodikara]]></dc:creator>
            <pubDate>Tue, 13 Nov 2018 11:36:40 GMT</pubDate>
            <atom:updated>2018-11-13T11:36:40.203Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*tViTYNQg1YIdJknnePnR6Q.png" /></figure><p>Today I’m gonna have quick look into how to integrate Quartz scheduler with Spring boot 2 and MySQL.</p><h4>What is the Quartz Job Scheduling?</h4><blockquote>Quartz is a <a href="http://www.quartz-scheduler.org/overview/features.html">richly featured</a>, open source job scheduling library that can be integrated within virtually any Java application — from the smallest stand-alone application to the largest e-commerce system. Quartz can be used to create simple or complex schedules for executing tens, hundreds, or even tens-of-thousands of jobs; jobs whose tasks are defined as standard Java components that may execute virtually anything you may program them to do. The Quartz Scheduler includes many enterprise-class features, such as support for JTA transactions and clustering.</blockquote><blockquote>Quartz is freely usable, licensed under the <a href="http://www.apache.org/licenses/LICENSE-2.0">Apache 2.0 license</a>.</blockquote><p><a href="http://www.quartz-scheduler.org/">Job Scheduler</a></p><p>I’m gonna create a sample spring boot application to show how this can be done in a simple way. I have used the “SPRING INITIALIZR” to initialize a sample maven project</p><p><a href="https://start.spring.io/">Spring Initializr</a></p><p>We can use the “spring-boot-starter-quartz” package to get quartz in to our project. To do this we need to add following dependency in to our pom.xml.</p><pre>&lt;dependency&gt;<br>    &lt;groupId&gt;org.springframework.boot&lt;/groupId&gt;<br>    &lt;artifactId&gt;spring-boot-starter-quartz&lt;/artifactId&gt;<br>&lt;/dependency&gt;</pre><p>This will include the most compatible quartz version according to our spring boot version. Since I’m using the latest spring boot version (2.1.0.RELEASE) we are gonna get quartz 2.3.0 to our application.</p><h3>Getting Started…</h3><p>After the project is created and updated we can start the development by creating the DB connection to the <strong>MySQL</strong>. Since I’m using spring boot we just need to add <strong>datasource </strong>properties to <strong>application.properties</strong> file,</p><pre>spring.datasource.platform=org.hibernate.dialect.MySQL5Dialect<br>spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver<br>spring.datasource.url=jdbc:mysql://localhost:3306/quartz_demo_db<br>spring.datasource.username=root<br>spring.datasource.password=admin<br>spring.jpa.open-in-view=<strong>false<br></strong>spring.jpa.show-sql=<strong>true</strong></pre><p>Next we can go-ahead and add quartz config properties to the property file.</p><pre>spring.quartz.job-store-type=<em>jdbc<br></em>spring.quartz.jdbc.initialize-schema=<em>never</em></pre><p>Quartz comes with its own in-built JobStores, and if you need you can create your own by implementing <strong><em>org.quartz.spi.JobStore </em></strong>interface. In this sample app I have used the <strong>jdbc </strong>JobStore. In spring-boot quartz provides two JobStores</p><ul><li>MEMORY— It keeps all of its data in RAM so once the application ends or crashes all the scheduling information is lost. Since it keeps its data in RAM, it is very fast and simple to configure</li><li>JDBC — JDBC JobStore keeps all of its data in a database via JDBC. Since it relies on database, configuration is a bit complicated and certainly is not as fast as RAM JobStore</li></ul><p>If you need to manage jobs on your own and modify them time to time without effecting the up-time of your application you should move to the <strong>JDBC </strong>job store. You can run <strong>clustered or non-clustered</strong> scheduler service when using the <strong>JDBC </strong>job store.</p><p>Next in the below are the other quartz properties, to add them to your app spring provides configuration property class <strong>QuartzProperties</strong>, and we need to add the prefix <strong><em>spring.quartz.properties </em></strong>to every property so the app can retrieve the properties.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/996/1*iYKEBvWsP3bWClvYnl64Kw.png" /><figcaption>Quartz Properties</figcaption></figure><p>Each Quartz Scheduler must has a unique ID. The value can be any string but must be unique for all schedulers. If we need we can AUTO generate this id by adding the property</p><pre>spring.quartz.properties.org.quartz.scheduler.instanceId=AUTO</pre><p>But in this example I have used a custom instance Id generator to generate a Id, by doing this if we are gonna create multiple instance of the schedulers we can have common strategy to generate instance Id to all the schedulers rather than having a random ID.</p><p><strong>CustomQuartzInstanceIdGenerator</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/564/1*YQpCwW7sB5rgcL1zVD8YNg.png" /></figure><p>this generates the instance Id on app startup</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*i7WNXBu-YqvjpZdHd-fpdw.png" /></figure><p>After all the properties are added to the <strong>application.properties</strong> we need to inject these properties to the spring quartz scheduler, for that I have implemented a spring configuration<strong> SchedulerConfig </strong>class to create <strong>SchedulerJobFactoryBean </strong>with those properties.</p><p><strong>SchedulerConfig</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/577/1*4-mUrmYVSTD1t6RQiqzFrg.png" /></figure><p><strong>SchedulerJobFactory</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/927/1*r0tna1UNyAwAJshSIGrMhg.png" /></figure><pre>@Override<br>protected Object createJobInstance(final TriggerFiredBundle bundle) throws Exception {<br>    final Object job = super.createJobInstance(bundle);<br>    beanFactory.autowireBean(job);<br>    return job;<br>}</pre><p>This will give <strong>Autowiring </strong>capability to the job beans.</p><p>Now we have a up and running quartz scheduler instance. Now we need to create jobs to schedules, in quartz we can create both cron schedulers and simple schedulers with repeat time limits. To create these schedulers first we need to create a <strong>Job </strong>and a matching <strong>Trigger </strong>to trigger the job.</p><p>We can create Cron Triggers using <strong>CronTriggerFactoryBean </strong>and Simple Triggers using<strong> SimpleTriggerFactoryBean. </strong>And <strong>JobDetailFactoryBean </strong>to create Jobs. I have created a custom Job schedule creator to create jobs and triggers.</p><p><strong>JobScheduleCreator</strong></p><p>Create new Job</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/912/1*Cp8QMbYX4XfMya6m5M89Tw.png" /></figure><p>Create Triggers</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/920/1*q5T7baOl65v7mkOO6oyQkQ.png" /></figure><p>Using <strong>JobScheduleCreator </strong>we can create both<strong> Jobs </strong>and <strong>Triggers </strong>for the scheduler.</p><p>In this application I have created a custom entity <strong>“SchedulerJobInfo” </strong>to keep track of the jobs. And also to use as a DTO to manage Jobs using <strong>SchedulerService. </strong>This contains if a job is a cron job or not, job class path, cron expression if cron job or repeat time if its a simple job.</p><p>And I have used the <strong>SchedulerServiceImpl </strong>to implement schedule jobs, update schedules, remove, pause, stop etc. These functions are provided by quartz scheduler and we can use them if we are using the <strong>JDBC </strong>jobs store.</p><p>I have also created two sample jobs one as a Cron job and the other as simple job.</p><p>In the <strong>SampleCronJob </strong>I have used a annotation <strong>@DisallowConcurrentExecution </strong>once this is added to a job and if we have multiple scheduler instance are running concurrently this job will not be executed by multiple schedulers concurrently.</p><p>And finally I have added a app startup handler to start all the pending jobs in the “<strong>scheduler_job_info</strong>” table. I have added this for testing purposes to start jobs in app startup.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/946/1*TC0WiXrDTkYsLRdT2Sk9mg.png" /><figcaption>source - <a href="http://www.javarticles.com/2016/03/quartz-scheduler-model.html">Java Articles</a></figcaption></figure><p>The <strong>full implementation </strong>and DB scripts of this can be found in github.</p><p><a href="https://github.com/ChamithKodikara/quartz-demo">ChamithKodikara/quartz-demo</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a8eaaf850805" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[JWT, OAuth2 authentication example with Spring Boot 2]]></title>
            <link>https://medium.com/@ChamithKodikara/jwt-oauth2-authentication-example-with-spring-boot-2-2e92bacd68e5?source=rss-2a4206fe48f6------2</link>
            <guid isPermaLink="false">https://medium.com/p/2e92bacd68e5</guid>
            <category><![CDATA[oauth2]]></category>
            <category><![CDATA[spring-boot]]></category>
            <category><![CDATA[java]]></category>
            <category><![CDATA[spring-security]]></category>
            <category><![CDATA[jwt]]></category>
            <dc:creator><![CDATA[Chamith Kodikara]]></dc:creator>
            <pubDate>Thu, 11 Oct 2018 09:13:00 GMT</pubDate>
            <atom:updated>2018-10-12T05:25:02.763Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*ZX3VsWVg8mNF1y1RXwlZTA.png" /></figure><h3>Spring Boot 2, OAuth2 and JWT authentication example</h3><p>In this article we’ll have quick look into how to implement OAuth2 login with use of JSON Web Tokens on spring cloud 2.</p><p>First we need to add spring cloud security dependencies to the project. Since I have used maven in this we need to update the pom.xml accordingly. (You can use “SPRING INITIALIZR” to initialize the project<a href="https://start.spring.io/)">)</a></p><p><a href="https://start.spring.io/)">https://start.spring.io/</a></p><pre>&lt;dependency&gt;<br>    &lt;groupId&gt;org.springframework.cloud&lt;/groupId&gt;<br>    &lt;artifactId&gt;spring-cloud-starter-oauth2&lt;/artifactId&gt;<br>&lt;/dependency&gt;<br>&lt;dependency&gt;<br>    &lt;groupId&gt;org.springframework.cloud&lt;/groupId&gt;<br>    &lt;artifactId&gt;spring-cloud-starter-security&lt;/artifactId&gt;<br>&lt;/dependency&gt;</pre><p>After all the dependencies are added we can start from configuring the Authorization Server. In here, I have only used one project for both Authorization server and resource server but you can implement these two on two separate projects.</p><p><strong>AuthorizationServerConfig</strong></p><p><a href="https://docs.spring.io/spring-security-oauth2-boot/docs/current/reference/html/boot-features-security-oauth2-authorization-server.html">2. Authorization Server</a></p><pre>@Configuration<br>public class AuthorizationServerConfig extends AuthorizationServerConfigurerAdapter {<br><br><br>    @Autowired<br>    @Qualifier(&quot;userServiceImpl&quot;)<br>    private UserDetailsService userDetailsService;<br><br>    @Autowired<br>    private AuthenticationManager authenticationManager;<br><br>    @Autowired<br>    private ApplicationProperties applicationProperties;<br><br>    @Autowired<br>    private AuthTokenEnhancer authTokenEnhancer;<br><br>    @Autowired<br>    private CustomClaimAccessTokenConverter customClaimAccessTokenConverter;<br><br>    @Autowired<br>    private DataSource dataSource;<br><br><br>    @Override<br>    public void configure(AuthorizationServerEndpointsConfigurer configurer) throws Exception {<br>        configurer<br>                .authenticationManager(authenticationManager)<br>                .userDetailsService(userDetailsService)<br>                .tokenServices(tokenServices())<br>                .tokenStore(tokenStore())<br>                .accessTokenConverter(accessTokenConverter());<br>    }<br><br>    @Override<br>    public void configure(ClientDetailsServiceConfigurer clients) throws Exception {<br>        clients.withClientDetails(new OauthClientDetailsServiceImpl(dataSource));<br>    }<br><br>    @Override<br>    public void configure(final AuthorizationServerSecurityConfigurer oauthServer) throws Exception {<br>        oauthServer.tokenKeyAccess(&quot;permitAll()&quot;).checkTokenAccess(&quot;isAuthenticated()&quot;).passwordEncoder(new BCryptPasswordEncoder());<br>    }<br><br>    @Bean<br>    public TokenStore tokenStore() {<br>        return new JwtTokenStore(accessTokenConverter());<br>    }<br><br>    @Bean<br>    public JwtAccessTokenConverter accessTokenConverter() {<br>        JwtAccessTokenConverter converter = new JwtAccessTokenConverter();<br>        KeyStoreKeyFactory keyStoreKeyFactory = new KeyStoreKeyFactory(new ClassPathResource(applicationProperties.getAuth().getKfName()),<br>                applicationProperties.getAuth().getKsPass().toCharArray());<br>        converter.setKeyPair(keyStoreKeyFactory.getKeyPair(&quot;demo&quot;));<br>        converter.setAccessTokenConverter(customClaimAccessTokenConverter);<br>        return converter;<br>    }<br><br>    @Bean<br>    @Primary<br>    public DefaultTokenServices tokenServices() {<br>        DefaultTokenServices defaultTokenServices = new DefaultTokenServices();<br>        defaultTokenServices.setTokenStore(tokenStore());<br>        defaultTokenServices.setSupportRefreshToken(true);<br><br>        TokenEnhancerChain tokenEnhancerChain = new TokenEnhancerChain();<br>        tokenEnhancerChain.setTokenEnhancers(<br>                Arrays.<em>asList</em>(authTokenEnhancer, accessTokenConverter()));<br>        defaultTokenServices.setTokenEnhancer(tokenEnhancerChain);<br><br>        defaultTokenServices.setReuseRefreshToken(false);<br>        defaultTokenServices.setAccessTokenValiditySeconds(applicationProperties.getAuth().getDefaultAccessTokenTimeout());<br>        defaultTokenServices.setRefreshTokenValiditySeconds(applicationProperties.getAuth().getDefaultRefreshTokenTimeout());<br>        defaultTokenServices.setClientDetailsService(new OauthClientDetailsServiceImpl(dataSource));<br><br>        defaultTokenServices.setAuthenticationManager(authentication -&gt; {<br>            UserDetails userDetails = userDetailsService.loadUserByUsername(authentication.getName());<br><br>            PreAuthenticatedAuthenticationToken token = new PreAuthenticatedAuthenticationToken(<br>                    authentication.getName(), authentication.getCredentials(), userDetails.getAuthorities());<br>            token.setDetails(userDetails);<br>            return token;<br>        });<br><br>        return defaultTokenServices;<br>    }<br>}</pre><p>In access token converter on <strong>AuthorizationServerConfig</strong> I have used a keystore key pair and custom access token converter</p><p>Keystore name and password are added to the <strong>application.properties</strong>, and <strong>demo.jks </strong>file is added to resource path</p><pre>app.auth.kfName=demo.jks</pre><pre>app.auth.ksPass=Za6{ApH5S}W4Pud</pre><p><strong>CustomClaimAccessTokenConverter</strong></p><pre>@Component<br>public class CustomClaimAccessTokenConverter extends DefaultAccessTokenConverter<br>        implements JwtAccessTokenConverterConfigurer {<br><br>    @Override<br>    public void configure(JwtAccessTokenConverter converter) {<br>        converter.setAccessTokenConverter(this);<br>    }<br><br>    @Override<br>    public OAuth2Authentication extractAuthentication(Map&lt;String, ?&gt; map) {<br>        OAuth2Authentication auth = super.extractAuthentication(map);<br>        auth.setDetails(map);<br>        return auth;<br>    }<br><br>}</pre><p>I have configured the Authorization server with client details and used the BCrypt encoder to encord client secret</p><pre>@Override</pre><pre>public void configure(ClientDetailsServiceConfigurer clients) throws Exception {</pre><pre>    clients.withClientDetails(new OauthClientDetailsServiceImpl(dataSource));</pre><pre>}</pre><pre>@Override</pre><pre>public void configure(final AuthorizationServerSecurityConfigurer oauthServer) throws Exception {</pre><pre>    oauthServer.tokenKeyAccess(&quot;permitAll()&quot;).checkTokenAccess(&quot;isAuthenticated()&quot;).passwordEncoder(new BCryptPasswordEncoder());</pre><pre>}</pre><p>Also I have added this sample Token Enhancer to the enhancer chain</p><pre>TokenEnhancerChain tokenEnhancerChain = new TokenEnhancerChain();</pre><pre>tokenEnhancerChain.setTokenEnhancers(</pre><pre>        Arrays.<em>asList</em>(authTokenEnhancer, accessTokenConverter()));</pre><pre>defaultTokenServices.setTokenEnhancer(tokenEnhancerChain);</pre><p><strong>AuthTokenEnhancer</strong></p><pre>@Component<br>public class AuthTokenEnhancer implements TokenEnhancer {<br><br>    @Override<br>    public OAuth2AccessToken enhance(OAuth2AccessToken accessToken, OAuth2Authentication authentication) {<br>        Map&lt;String, Object&gt; info = new HashMap&lt;&gt;();<br>        info.put(&quot;TEST&quot;, &quot;TEST Enhancer&quot;);<br>        ((DefaultOAuth2AccessToken) accessToken).setAdditionalInformation(info);<br><br>        return accessToken;<br>    }<br><br>}</pre><p>Next we’ll look into <strong>ResourceServerConfig.</strong></p><p><a href="https://docs.spring.io/spring-security-oauth2-boot/docs/current/reference/html/boot-features-security-oauth2-resource-server.html">3. Resource Server</a></p><pre>@Configuration<br>public class ResourceServerConfig extends ResourceServerConfigurerAdapter {<br><br>    @Autowired<br>    private DefaultTokenServices tokenServices;<br><br>    @Autowired<br>    private TokenStore tokenStore;<br><br>    @Autowired<br>    private ApplicationProperties applicationProperties;<br><br>    @Override<br>    public void configure(ResourceServerSecurityConfigurer configurer) {<br>        configurer<br>                .resourceId(applicationProperties.getAuth().getResourceId())<br>                .tokenServices(tokenServices)<br>                .tokenStore(tokenStore);<br>    }<br><br>    @Override<br>    public void configure(HttpSecurity http) throws Exception {<br>        http<br>                .cors().and()<br>                .csrf().disable()<br>                .authorizeRequests()<br><br>                .antMatchers(applicationProperties.getEndpoint().getTestEndpoint()).hasAuthority(&quot;test&quot;)<br><br>                .antMatchers(applicationProperties.getEndpoint().getIndex()).permitAll()<br>                .antMatchers(&quot;/**&quot;).denyAll();<br>    }<br>}</pre><p><strong>Enabling Auth server and resource server</strong></p><p>After we config both Authorization server and resource server we need to enable both. To this we can use @EnableAuthorizationServer &amp; @EnableResourceServer annotations on Main class. Then all of your APIs will be secured with Oauth2 and spring security.</p><pre>@EnableAuthorizationServer<br>@EnableResourceServer<br>@SpringBootApplication<br>public class OauthDemoApplication {<br><br>    public static void main(String[] args) {<br>        SpringApplication.<em>run</em>(OauthDemoApplication.class, args);<br>    }<br>}</pre><p>The <strong>full implementation </strong>and DB scripts of this can be found in github</p><p><a href="https://github.com/ChamithKodikara/oauth-demo">ChamithKodikara/oauth-demo</a></p><p>This is an Intellij based project, so it should be easy to import and run using Intellij.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2e92bacd68e5" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>