This is the second part of the series on hapi.js, the first part explained what hapi is all about and basics of its plugins and request lifecycle along with a piece of code demonstrating how to create a “hello world” hapi server. This part explains about a starter-kit prepared by me using hapi v18.x which can help save a substantial amount of development effort towards doing the foundational work for a scalable web application.

The Starter-Kit

Most modern web applications need two front-ends for users — one that’s browser based (for desktops/laptops) and another which is usually a (native or hybrid) mobile application. Browser-based frontends are typically used for administrative purposes (admin portals) and have limited features compared to their mobile counterparts, which are generally designed for end-users.

The concept behind this starter-kit was to provide developers with a way to satisfy both of these needs by having the same hapi server cater to both kinds of consumers, while reducing the redundancies of controllers and routes at the same time.

GitHub Repo

The codebase for this writeup can be cloned from the following GitHub repo:

khandelwal-arpit/hapijs-starterkit

Starter-Kit Details

This starter-kit serves as a great starting point for all the hapi developers who were looking for a hapi based platform as their production server. Once successfully deployed, this kit gives developers access to the following two modes of interaction with the server:

1. http://localhost:<port> — Serves web requests
2. http://localhost:<port>/api — Serves rest api requests

Web and REST APIs both have their independent authentication mechanisms. This kit assumes the availability of ‘MongoDB’ installation on the localhost where the server will run or the use of docker-compose to boot up a mongodb container and link the application with it within the realm of docker.

The starter-kit contains basic features such as user management with login, logout, password reset, profile view, dashboard view etc. Forgot password implementation is done using ‘nodemailer’ (sends an email with reset password link with a unique token that expires in an hour). All the environment variables are managed using ‘dotenv’ node package for development, the same will need to be configured at the host server in the production environment.

Libraries Used

The starter-kit uses the following primary libraries for satisfying various requirements:

• Hapi — Server side framework
• Docker — Docker container support
• Handlebar — HTML templating engine
• Mongoose — Mongo database ORM
• SASS — CSS preprocessor
• WebPack — Asset pipeline
• Dotenv — Environment variable emulator
• Good — Logger mechanism
• JWT — Authentication mechanism for APIs
• Config — Configuration Handler
• Nodemailer — Module to send emails
• Mocha — Unit test framework

Opinionated Application Structure

The kit is designed in such a way that all the relevant artefacts are organised in their separate folders. You will see app, assets and config being the topmost container directories along with the root directory. A detailed view of the application structure is specified below:

├── app
│   ├── controllers
│   │   ├── api            // Controllers
│   │   └── web            // Controllers
│   ├── helpers            // Helpers
│   ├── models             // All mongoose models are defined here
│   ├── routes             // All app routes are defined here
│   │   ├── mobileApi      // RESTAPI routes for multiple versions
│   │   │   ├── v1         // Routes for version1
│   │   │   └── v2         // Routes for version2
│   │   └── webApi         // WEB api routes.
│   └── templates          // Server-rendered handlebar templates
├── assets                 // Contains all static resources 
│   ├── fonts              // Fonts used in application
│   ├── images             // Images used in application
│   ├── scripts            // Scripts used in application
│   │   ├── js             // User defined scripts
│   │   └── vendor         // Vendor scripts
│   └── styles             // All SASS stylesheets
├── config                 // Contains all app configurations
│   ├── assets.js          // Assets configuration file
│   ├── config.js          // Application configuration 
│   ├── default.json       // Configuration file
│   ├── manifest.js        // App manifest file
│   ├── meta.js            // App metadata file
│   └── ssl                // Contains ssl certificates
├── lib                    // Core application lib/plugins
├── logs                   // Contains app log file 
├── .gitignore             // standard git ignore file
├── .babelrc               // Babel config
├── .eslintrc              // Define eslint rules
├── .eslintignore          // Ignores certain files for eslint rules
├── Dockerfile             // Standard docker file
├── docker-compose.yml     // Standard docker compose file 
├── server.js              // Contains all app configurations
├── .env                   // dotenv configuration
└── test
    ├── testcases          // Test cases organised by module names
    └── test.js            // Test file

The best thing about using opinionated structures is that there is basically one way (the right way) to do things and trying to do it differently will be difficult (and frustrating). Doing things the right way can make it very easy to develop with the software as the number of decisions that you have to make is reduced and the ability of the software designers to concentrate on making the software work is increased. This kit follows the same principle, I have used this kit in a variety of projects with a great amount of success due to the homogeneity of the code foundation and only variable being the business logic.

If you consider development in a big team environment then opinionated frameworks like these actually help you a lot as a technical lead. One thing you are always sure is about the quality of the code being written and the way different pieces join hands to combine into a big application. Without such frameworks, code review can be a nightmare considering there are global teams with developers having different thought process and ability to adapt certain coding styles.

Application Flow

Let’s walk ourselves through the signup flow in the codebase to understand how the different directories, plugins and scripts come into picture for a POST api request for signup from a mobile device.

The first point of contact in the application is the routes directory. It contains the routing information for the various HTTP requests. All the routers are configured as independent plugins, here is how the source code of the routes/mobileApi/v1/signup.js plugin looks like:

signup.js (router)

Whenever a POST request for signup is sent to the server the, ‘signup_routes_v1’ plugins catches it and contacts the /controllers/api/signup controller for handling the request.

signup.js (controller)

Validating data can be very helpful in making sure that our application is stable and secure. hapi allows this functionality by using the module Joi, which allows us to create our validations with a simple and clear object syntax. In this case the validate routine inside the userSignUp function does the same by applying rules on the email field, password and confirm password fields and the name field. If for any reasons the applied validations fail, a HTTP 400 response is immediately returned from the server.

Once the validations pass, the async handler function comes into picture. It is dependent on a helper /helpers/signup.js which is defined in the helpers directory at the root.

signup.js (helper)

The helper takes help from the User.js mongoose model to search if the passed user details match an existing user of the application, if it does then a HTTP 409 response is sent back to the caller which signifies that this is an existing user and system cannot allow its creation. However, if the details don’t match any existing user in the mongodb, then the save method results in creating one and returns a HTTP 201 stating the signup was successful.

This entire flow could be tested using the swagger documentation as follows:

The other API flows are very similar to the signup flow, hopefully the illustrations above have given you fair bit of insight on understanding the code base better.

Authentication

Both of the servers spun up in this kit have independent authentication measures in place. The web application server uses cookie-based authentication, while the API server uses JWT authentication for rest API access. The deployment and application management is done via standard node scripts, webpack and forever.

Mobile api authentication request handler:

Web authentication request handler:

Glued Server

A question you might be having right now is where and how the hapi server has been defined. The answer is, its not defined in the traditional manner which most tutorials suggest you to do:

const Hapi = require(‘hapi’); 
const server = Hapi.server({ 
      port: 3000, 
      host: ‘localhost’
});

Rather, the server is defined inside the server.js file as follows:

Glue is a plugin that works with hapi to configure a server relatively easy, it works by exporting a single function named compose to your hapi server. Inside of compose are two parameters that work to configure your server automatically, from server connections to server plugins.

The manifest file contains declarations of plugins created throughout the codebase along with the configurations passed to them declaratively inside JSON objects. We will discuss more about the configurations in the next section, for now all you need to know is we are using Glue to create the hapi server, glue needs a pre defined (and of course opinionated) config file (called as manifest.js in this case) and then you can simply invoke the server.start() method to boot the hapi server.

Configurations

There are two important configuration files present in the starter-kit:

1. config/config.js
2. config/manifest.js

config.js

The config.js file contains the configurations needed by various plugins defined in the codebase. For ex. the mongoose json object contains the mongodb connection uri according to the environment (default, test, prod), the email object contains the various settings needed to send an email from our starter-kit etc

The manifest.js file on the other hand contains the plugin declarations and provides them with the configuration needed defined in the config.js.

manifest.js

For registering any new route using the structure explained so far, the last thing you will need to do is register the plugin in this manifest so that Glue can compose the hapi server with the set of plugins at runtime. If you have defined the plugin properly but forgot to include it here, Glue has no way of knowing that your new plugin needs to be registered with hapi and hence it won’t show up.

Unit Testing

If you are using the Chrome browser for unit testing the application, you can use the ‘node-inspector’ plugin pre-configured with this app for debugging purposes. This kit supports writing unit test cases using ‘Mocha’ and ‘Supertest’ node packages. To execute the test cases please run the following command:

# Test the server
$ npm test

Logging

The logging aspect for this kit is covered with ‘Good’ which is a hapi plugin for logging. The configurations are organised using ‘Glue’ along with ‘Confidence’ (as explained earlier) which makes it very easy to switch between different plugins for the various functionalities while keeping the hapi’s ‘configuration over convention’ theme intact.

We have used the following configuration for the logging aspects of the starter-kit using Good (defined in config.js):

“myConsoleReporter” configuration above instructs hapi to log the request details on the console (stdout), whereas the “myFileReporter” tells hapi to log the same set of details in the /logs/log file.

Swagger Documentation

The APIs are well documented for unit testing and sharing with UI team using Swagger. Please use the following URL for accessing the same:

http://localhost:8000/documentation#/

Standard hapi Packages Used

The following packages/plugins were used from the standard hapi.js ecosystem in the development of this kit:

1. Crumb
2. Glue
3. Good
4. Good-Console
5. Hoek
6. Hapi-auth-cookie
7. Inert
8. Yar
9. Wreck

Webpack v4

As you might already know, Webpack is an open-source JavaScript module bundler. Its main purpose is to bundle JavaScript files for usage in a browser, yet it is also capable of transforming, bundling, or packaging just about any resource or asset.

In case of this starter-kit too, we have used webpack to create following two important files:

1. bundle.js
2. vendor.js

Along with these files, webpack also bundles the entire css scripts present in the development folders into a single file “index.css” and dumps all of them in the .build directory which gets created once webpack runs as a prestart script for the server execution. We have used v4 for this kit and the webpack configuration looks as follow:

webpack.config.js

Running the Server Locally

To run the server in standalone mode use either of the following commands:

# Install dependencies
$ npm install

# Run the node server in dev mode
$ npm start

# Run the node server in production mode
$ npm run prod

Running the Server in Docker Container

Please ensure that docker and docker-compose are installed on your system.

The contents of the Dockerfile are as follows:

The docker-compose file is organised as follows:

Steps to run app in docker container:

# cd to project dir

# Create build
$ docker-compose build

# Start the server in daemon thread
$ docker-compose up -d

# Stop the server
$ docker-compose down

In case you are willing to use this project as is (i.e. without external mongodb installation), the docker-compose provided with this project should suffice. It brings along a mongodb service which stores the data in the /db/data directory. But in case you wish to use your existing MongoDB installation then please remove the mongodb service from the docker-compose.yml file and correct the database_url environment variable.

Application Screenshots

Here are some of the screenshots of the default features being provided along with the starter-kit:

Login Page

Sign-up Page

Forgot Password Page

Dashboard View

Settings View

Conclusion

As you can see, the starter-kit is designed to save you hours of coding time by giving you a streamlined, efficient and clean-coded interface and architecture to base your own development on. Try it out and be sure to let me know your feedback.

Hapi coding everyone!

If it was interesting or helpful to you, please do press the 👏 clap button and help others find this story too.

Did you know you could give up to 50 claps?

Hapi.js — Project Structure and Best Practices (Part 2) was originally published in The Resonant Web on Medium, where people are continuing the conversation by highlighting and responding to this story.

Preface

Node.js has unarguably become the backbone of the software industry. The increased popularity of JavaScript over the last few years has propelled Node forward in terms of production usage. Perhaps for the first time, JavaScript and Node offer the opportunity to develop entire n-tier applications using a single language.

As it is with all web frameworks, Node provides the essentials for an application — request and response objects, methods to manipulate HTTP requests, ability to interact with a database using popular ORM tools, robust production deployment libraries and more. “Pure” Node apps are stunningly fast, but they lack in the supporting cast of middleware, routing, and plugins that reduce the amount of code needed for a modern web application.

This is where web frameworks shine. This blog post is about hapi.js which is a framework with one of the fastest growing user base, the primary reason being the stability and scalability that hapi provides. Let’s explore more about this wonderful framework in the coming paragraphs.

What is hapi.js?

hapi.js (short for Http-API, pronounced happy and also known as hapi) is an open-source framework for developing scalable web applications. One of the most basic use case of hapi is to build REST APIs. You can build application programming interface (API) servers, websites, and HTTP proxy applications with hapi.js.

It was created by the mobile team at Walmart Labs — led by Eran Hammer to handle their traffic for events like Black Friday, one of the busiest days for online shopping in the U.S. calendar.

Why is there so much of fuss about hapi?

To give you an idea of just why the developer community believes hapi is so robust and dependable, it was able to successfully manage millions of shopping transactions, returns and more at Walmart during Black Friday, arguably the busiest retail shopping day of the year in the U.S.

hapi was able to handle all of Walmart mobile Black Friday traffic with about 10 CPU cores and 28Gb RAM (of course we used more but they were sitting idle at 0.75% load most of the time). This is mind blowing traffic going through VERY little resources. — Eran Hammer

The original versions of hapi used the Express framework. Walmart found that there were limitations with Express that made the framework unsuitable for their particular requirements. Express was missing some key features, so Walmart eventually had hapi evolve to its own stand-alone framework.

Here is a list of some of the great names using hapi in production today:

Building Blocks

Throughout Walmart’s journey of rebuilding their mobile services tier and working with other frameworks, they realised that certain values were vital to building applications in Node for keeping the code intuitive, maintainable, and scalable. As a result, hapi development team centred around the following concepts:

1. Building value, not infrastructure
2. Configuration is better than code
3. Separation of business logic from the transport layer
4. Open source and community-centric
5. Ecosystem (Opinionated nature, plugins)
6. Small modules

Plugins

hapi provides a robust plugin system that allows you to add new features and fix bugs at a fast pace. There are dozens of plugins for hapi, ranging from documentation to authentication, and much more. Some of the most important plugins are listed here.

The benefits of splitting code into plugins reaches beyond just your own code, but easily allows users to share code that is beneficial to everybody. For example, you may have written code specific to Facebook authentication (bell) or a plugin to do something as simple as displaying the routes on startup (blipp).

A benefit of the plugin model and having replaceable parts in hapi is that you’re not bound to a particular way of doing things. If there later comes along a better plugin than the one you’re currently using, then switching it out is far easier than if that part of the code is tightly coupled with the code you want to keep. This applies not just to third-party plugins but anything within your own system as well.

When large companies work on plugins across multiple teams, things like caching can be determined on a per-plugin basis. The teams don’t have to communicate with each other, and even though the plugins run as part of the same server, routes in the different servers can be configured via the plugin to use, for example, different caching mechanisms that are suitable for what that particular piece of code is doing.

Express vs hapi

Both Express and hapi aim to be highly flexible, simple, and extensible. This similarity means that both have easy-to-use APIs, they’re highly modular, and can support your application as it grows potentially very large. The learning curve for these frameworks, since they are quite straightforward, is low, unlike a more opinionated framework like Meteor. If you are coming from using Express you should be able to quickly pick up Hapi, and vice versa.

There are, however, some philosophical differences between the two frameworks.

• Hapi.js includes more “batteries” by default than Express does. For instance, when parsing payload from forms to via “POST” requests, with Express, you have to include the body-parser middleware. You then use that to parse the POST payload and use the data from the user. On the other hand, with Hapi, you need no middleware. The payload is parsed for you by the framework itself, and you can access the payload directly on the request object. Little conveniences like that are replete in Hapi.
• “Middleware” is the name given to software modules that work on HTTP requests in succession before the final output is returned to the user in a response. Express lets you attach middleware to handle each request. Hapi, however, works with plugins that provide middleware functionality.
• Serving static files is remarkably similar between Hapi and Express. Hapi uses the Inert plugin, whereas Express uses the express.static built-in middleware.

Hapi makes sense if you have a well-defined app idea that fits within sensible defaults for the programming approach. Hapi takes an opinionated approach to things like app security, and its plugins system is not as extensive as the Express middleware ecosystem. If you are a large team, the conventions of Hapi can come in useful to keep your code maintainable. Express, on the other hand, works best if you are after flexibility and making most development decisions on your own.

Request lifecycle

hapi is specifically designed to work in complex enterprise environments where multiple teams work on individual projects or overlapping pieces that must connect into one seamless result. By enabling developers to work on modular, reusable code without being narrowly constrained, developers can concentrate on the core project without trying to erect meta-frameworks or refactor the code.

To show this process in action, consider the typical hapi request lifecycle:

Here you can see that the developer is provided with hooks, in the form of handlers and extensions, to implement the logic. hapi largely stays out of what goes into these hooks. As project complexity grows, the need to decouple functionality and distribute the load also grows. The Hapi server becomes the outward-facing interface (by way of an API or UI), while leveraging a wide array of other technologies to break down monoliths of business logic into smaller, bite-sized pieces (which can in turn be managed by Hapi).

hapi 16 v 17

A quick side note about which version you should use. There was a substantial shift in the framework from 16 to 17 (latest being v18.x), where it went from using callbacks to async/await. This blog post, and the next one (with the starter-kit), will use hapi v18.

A Simple Server

A simple hapi based api server can be coded in a few lines of code as follows:

Save the above as server.js and start the server with the command:

node server.js

Now you'll find that if you visit http://localhost:3000 in your browser, you'll see the text “Hello, world!”, and if you visit http://localhost:3000/node you'll see “Hello, node!”.

Part 2

Thanks for reading the first part from a series of two articles on hapi. If this is enough to raise your interest in using this adaptable framework, be sure to check out the second part here.

It contains a detailed walkthrough of the hapi based starter-kit app built by me, which provides an excellent starting point for all hapi developers to use in their own production environments.

If it was interesting or helpful to you, please do press the 👏 clap button and help others find this story too.

Did you know you could give up to 50 claps?

An introduction to Hapi.js (Part 1) was originally published in The Resonant Web on Medium, where people are continuing the conversation by highlighting and responding to this story.

This is the second article in the series of two, the first part concentrated on the theoretical aspects of Kubernetes. The aim of this story is to allow the readers to spin up their own fully functional Kubernetes cluster with a sample micro services based application deployed on it. The idea is to cover most of the Kubernetes objects and best practices those should be considered while setting up a new cluster.

Philosophy

I started learning Kubernetes almost an year ago after wrapping up my head around Docker. According to Google, Kubernetes is a portable, extensible open-source platform for managing containerised workloads and services, that facilitates both declarative configuration and automation.

Honestly, the only simple thing I could get my hands on for first few weeks was the definition of it. No matter how hard I tried to glue things together to understand this exciting technology, I kept on failing and for a very simple reason that I was trying to too hard by looking at different articles and blog posts and was trying to accomplish all at once, however I wasn’t building something around it which is unarguably the best approach to learn a new tech, yes, do it!

Its only then, that I embarked on a quest to make sure not only that I learn the concepts of Kubernetes myself, I do it in a way that makes it easier for my colleagues to understand and learn it faster than I did. The current starter-kit takes the approach of ‘read-it’ first along with ‘do-it’ next and encapsulates another hot topic ‘micro services’ inside the kit that will help a lot of techies to deploy and scale their micro services based deployment using docker containers and kubernetes.

Application

The application built as a part of this quest is a simple browser based calculator whose front end is written using Vue.js and the operations (addition, multiplication etc) are served by following micro services written in various programming languages:

1. Expressed service (Express.js)
2. Happy service (Hapi.js)
3. Bootstorage service (Spring Boot)
4. Vuecalc service (Vue.js)

The Expressed and Happy services are rest api based implementations not doing much tangibly but only serve basic calculator functions as APIs. The Bootstorage micro service acts as a backend service that stores and retrieves data provided to it via the former two.

The Vuecalc service is a frontend application designed in Vue.js. It resembles a calculator, two of whose primary operations i.e. Add and Subtract are served by the expressed service and the other two operations, i.e. multiply and division are served by the happy service.

Along with the micro-services, following components are deployed in the cluster as well:
1. Ambassador API Gateway
2. Redis (data store)

Ambassador(provided by datawire.io) is an open source Kubernetes-Native API Gateway built on the Envoy Proxy which is a native kubernetes api gateway. Redis acted as a data store for this implementation, storing the operations done on the calculator and serving them on on demand.

GitHub Repo

The codebase for this story can be found in the following repo:

khandelwal-arpit/kubernetes-starterkit

Architecture

The complete architecture of this implementation can be summarised as:

As we can see, Ambassador is acting as a single point of entry in the Kubernetes cluster, channeling all the incoming requests to appropriate services. The various microservices are deployed at the following addresses inside the cluster:

Each one of the microservices has liveness and readiness probes inbuilt for the cluster to determine their health and update the users about any potential downtime. The probes are configured for each service at the following endpoints:

These probes are configured in the individual deployment configurations as follows:

All the services are currently deployed under the default namespace to keep things simple. However we strongly suggest that users should opt for different namespaces specially if they are planning to deploy their DEV, QA and UAT environments all on the same cluster (of course assuming PROD is going to be an independent cluster).

Services

The different services created for this starter-kit are summarised in the following diagram:

Vuecalc

This is a vue-cli 3.0 based Vue.js application which renders the calculator UI as shown below:

The service is composed of ‘home’, ‘calculator’ and ‘history’ views. It uses Axios library to do all the API calls to the kubernetes cluster. It uses a sound architecture with separate components and services for different aspects of the interface. The ‘history’ view shows a list of recent operations done over the calculator application by fetching them from the ‘Bootstorage’ service located inside the cluster.

Expressed

It is an Express.js (v4.16.x) based web application serving basic APIs for addition and subtraction. It is a containerised service and deployed inside the cluster as a pod eligible to horizontally scale under extreme conditions. The Vuecalc service calls the apis from this service when it has to do an add or subtract operation, once the operation is complete an async call is placed to the Bootstorage service to store the last performed operation in the Redis data store.

Happy

It is a Hapi.js (v17.8.x) based application having two basic APIs for multiplication and division. It is also a containerised service and similar to the Expressed service, is deployed internally in the kubernetes cluster for serving the apis. It also places an async call to the ‘Bootstorage’ service to store the last executed operation on Redis.

Bootstorage

It is a Java (v1.8) and Spring Boot (v2.x) based microservice that has two primary purposes:

1. Store executed operations inside Redis, and
2. Return the list of operations stored in Redis sorted in the order of execution (last operation at top).

Redis has been used in the cluster as a statefulset which has persistence enabled and uses persistent volume claim templates in the cluster to store the Redis dump files which allows the cluster to make sure Redis always retains the values in between cluster outages too. The Redis connection parameters are dynamically passed to the pod from the ENV variables which are provided to it using the config maps.

Kubernetes Components

Following kubernetes components were used during the development of this starter kit:

1. Services
2. Deployments
3. Pods
4. StatefulSets
5. Cronjob
6. Secrets
7. Config-Maps
8. Persistent Volumes
9. Persistent Volumes Claims
10. Storage Class
11. Kubernetes Dashboard

We are assuming that there is an existing kubernetes cluster already running on one of the renowned cloud hosting service provider -

1. Google cloud (Kubernetes Engine)
2. MS Azure (Kubernetes Cluster Service)
3. AWS (AKS)

Or, if this is not the case, you can try Docker-for-desktop, it allows us to create a sample kubernetes cluster with a single node inside it. Even though it only has a single node out of the box, it suffices the complete development and testing effort needed to run an application on kubernetes cluster before we actually go to the cloud. We suggest you try your configurations first locally and once it works as expected then move to the cloud to save unnecessary hosting cost.

Some of the important components used for current project’s development are explained in sections below.

Config Maps

ConfigMaps allow us to decouple configuration artifacts from image content to keep containerized applications portable. For the purpose of demonstration, the expressed and happy service both use a config map of their own to pass ENV var log_levelto them. The yaml look as follows:

expressed-cmap.yaml

happy-cmap.yaml

These configurations are referred inside the deployment specifications as follows:

expressed.yaml

StatefulSet

Like a Deployment, a StatefulSet manages pods that are based on an identical container spec. Unlike a Deployment, a StatefulSet maintains a sticky identity for each of their Pods. These pods are created from the same spec, but are not interchangeable as each has a persistent identifier that it maintains across any rescheduling.

A StatefulSet operates under the same pattern as any other Controller. We define our desired state in a StatefulSet object, and the StatefulSet controller makes any necessary updates to get there from the current state.

StatefulSets are valuable for applications that require one or more of unique network identifiers or persistent storage or ordered, graceful deployment and scaling. In our case we had the need to store the data for various operations happening in the calculator application so that we can show them when user goes to the history section of the application. Here is how the configuration yaml for it looks like:

redis.yaml

The Redis runs on default port 6379 however its access is password protected and needs the password calculator for any access within the cluster. The appendonly command instructs Redis to create a dump file and append to it every 900 seconds if single edit has been made or every 30 seconds if 2 edits have been made.

The volume that gets mounted points us to a volume claim which is being dynamically allocated to us using the persistent volume claim template (pvc template) definition. The pvc-template instructs the cluster to create a storage on disk and allocate the same to Redis each time it reboots. This way we do maintain the state of Redis deployment which is crucial since it is acting like the data store for our use case. You can easily replace it with MySql or any other database of your choice, but please bear in mind that you will need to do certain modifications to the Bootstorage service code in order to support the new storage mechanism.

Cronjob

A Cron Job creates Jobs on a time-based schedule. One CronJob object is like one line of a crontab (cron table) file. It runs a job periodically on a given schedule, written in Cron format.

In our case, we have written a cronjob that invokes the following API every 15 minutes:

http://bootstorage-svc:5000/api/bootstorage/deletelru

The purpose of this api (written in the Bootstorage service) is to delete the oldest operation that was performed on the calculator. This way we keep the footprint of our data store small and at the same time have a perfectly valid use case to show how a cronjob can be executed in the application to do a recurring operation. The yaml for the same looks as follows:

The parameter successfulJobsHistoryLimit makes sure that we see exactly 3 entries of the last successful executions of the cronjob in our kubernetes history either viewed via kubectl or on the dashboard. This cronjob also presents a classic example where a task defined in one of your micro service needs to execute periodically and you were hesitant to create an entire pod just for the sake of running the task, using this approach you can easily instruct any running pod via the API call which is executed inside a ~35MB alpine container spun up at runtime.

API Gateway

Ambassador is a production tested, open source API Gateway that exposes the power of Envoy Proxy in Kubernetes. Ambassador relies entirely on Kubernetes for reliability, availability, and scalability. For example, Ambassador persists all state in Kubernetes, instead of requiring a separate database. Ambassador has virtually no moving parts, and delegates all routing and resilience to Envoy Proxy and Kubernetes, respectively. It integrates the features teams need for microservices, including authentication, rate limiting, observability, routing, TLS termination, and more.

In our case, we decided to deploy Ambassador service as a LoadBalancer listening on port 80. The configuration that was responsible for the routing of incoming calls to independent micro services is as follows:

For services such as Expressed, Happy and Bootstorage we are rewriting the incoming request path to /api/* and the service mappings are following the convention of service-name:port where the ports are those allocated to these services inside the cluster. There are different ways how developers use the routing configurations, some of us prefer to put this information as a kubernetes-ambassador annotation inside the respective deployment configurations but we personally like to keep them all at a single place inside the ambassador configuration file and that’s exactly what you will notice in the configs.

Kubernetes Dashboard

Dashboard is a web-based Kubernetes user interface. You can use Dashboard to deploy containerised applications to a Kubernetes cluster, troubleshoot your containerised application, and manage the cluster resources. You can use Dashboard to get an overview of applications running on your cluster, as well as for creating or modifying individual Kubernetes resources (such as Deployments, Jobs, DaemonSets, etc).

Please follow this link to learn more about how to set up the dashboard for your cluster and make use of a decent web interface to administer the kubernetes cluster. Here are a few screen shots of how the calculator application components look like on our dashboard after successful deployment:

Overview

Resources

Services

Deployments

Pods

StatefulSets

Cron Jobs

Config And Storage

Build and Deploy

There are some assumptions that we are making (before you can run the build and deploy command) like you have the following pre-installed on the node/environment where this cluster is being set up -

1. Docker
2. Maven
3. NPM
4. Kubectl

Once these are setup, please execute the following command from the root directory of this repo:

sh build-deploy.sh

This command is equipped to prepare all the docker containers needed for the setup of this application. It will create them all one by one and finally execute the configurations placed in the k8s directory which will result in the creation of various artifacts on the kubernetes cluster.

Important Kubernetes Commands

Here are some basic commands for using Kubernetes command line utility:

1. kubectl proxy (Run kubernetes dashboard)
2. kubectl describe secret admin-user-token-6kchd -n kube-system (Generate token)
3. kubectl create -f k8s (Deploy to kubernetes)
4. kubectl get deployments (Check existing deployments)
5. kubectl get services (Check existing services)
6. kubectl set image deployments expressed expressed=expressed:latest (Update image for Deployment)
7. kubectl rollout status deployments expressed (Check status of updated rollout)
8. kubectl scale — replicas=0 deployment express-svc (Scale down any deployment)
9. kubectl scale — replicas=1 deployment express-svc (Scale up a deployment)
10. docker image prune (Remove all dangling images)

Conclusion

With this we now come to conclusion of the second part and the series on Kubernetes. I hope both the articles made sense to the readers and that it helps them gain a better grip on this exciting new technology and at the same time allows them to have a kick start without investing a plethora of hours just to set Kubernetes up.

As always, looking for your feedback!

If it was interesting or helpful to you, please do press the 👏 clap button and help others find this story too.

Did you know you could give up to 50 claps?

How to deploy Microservices based application on Kubernetes? was originally published in The Resonant Web on Medium, where people are continuing the conversation by highlighting and responding to this story.

It might be easy to deploy a container, but operationalising containers at scale, especially in concert with microservices — is not for weekend enthusiasts. It requires planning, and most experts say an orchestration tool is a must.

That’s the point at which the conversation will likely turn to Kubernetes. The platform was first developed by a team at Google, and later donated to the Cloud Native Computing Foundation (CNCF). It’s not the only option for container management, but it has rapidly become one of the most popular one.

I have been using Kubernetes for last twelve months or so and I must say I am quite impressed with the way our microservices based applications are being managed now. Earlier, some of the biggest challenge we faced were:

• Remembering where our containers are deployed.
• Scaling them manually or taking help of cloud provider’s configurations.
• Being so tightly coupled with the cloud provider that migrating the environment to a more economical provider was a nightmare.
• Dependence on cloud provider’s API gateway.
• Not being able to utilise the VMs to their full potential due to manual orchestration and having to do the math around RAM and processor consumption on the VMs.
• Inability to use some of the industry best-practices such as circuit breakers, rater limiters, uniform logging at each VM etc.
• Issues with reusing the same VMs for different environments such as DEV, QA, UAT and PROD.
• Canary releases of updated containers.

With the introduction of Kubernetes clusters, most of the challenges listed above have now been overcome with least amount of efforts. Kubernetes is an opinionated solution to these issues but the best part is it really doesn’t matter if you are deploying the cluster over AWS or Azure once the configurations are established. Also, managing which container goes where is no longer our headache, Kubernetes control plane takes over such tasks and does them very efficiently.

Throughout this series I will try to emphasise on the above mentioned challenges and how Kubernetes helped me solve them, also I will introduce a starter-kit that I have prepared for beginners to learn Kubernetes from Zero. Hopefully by the end of second part you will have fair bit of information and a good starting point into the Kubernetes wonderland!

What does Kubernetes mean?

The name Kubernetes originates from Greek, meaning helmsman or pilot, and is the root of governor and cybernetic. K8s is an abbreviation derived by replacing the 8 letters “ubernete” with “8”.

Its an open source platform that automates Linux container operations eliminating many of the manual processes involved in deploying and scaling containerised applications. Kubernetes provides a platform to schedule and run containers on clusters of physical or virtual machines. More broadly, it helps you fully implement and rely on a container-based infrastructure in production environments. And to do so in a way that automates many operational tasks.

In other words, you can cluster together groups of hosts running Linux containers, and Kubernetes helps you easily and efficiently manage those clusters.

What does Kubernetes do?

As a Kubernetes user, you can define how your applications should run and the ways they should be able to interact with other applications or the outside world. You can scale your services up or down, perform graceful rolling updates, and switch traffic between different versions of your applications to test features or rollback problematic deployments. Kubernetes provides interfaces and composable platform primitives that allow you to define and manage your applications with high degrees of flexibility, power, and reliability.

Kubernetes eliminates all of the manual processes involved in the deployment and scaling of containerised applications. It’s capable of clustering together groups of servers hosting Linux Containers while also allowing administrators to manage those clusters easily and efficiently.

Kubernetes makes it possible to respond to consumer demands quickly by deploying your applications within a timely manner, scaling those same applications with ease and seamlessly rolling out new features, all while limiting hardware resource consumption. It’s extremely modular and can be hooked into by other applications or frameworks easily. It also provides additional self-healing services, including auto-placement, auto-replication and auto-restart of containers.

Kubernetes vs Docker Swarm

There also exists Docker’s own platform called Swarm. It accomplishes much of the same tasks and boasts a lot of the same features. The primary difference between the two is that Swarm is centralised around the use of Docker, while Kubernetes tends to adopt a more generalised container support model.

Docker and Kubernetes can actually work together. As far as management platforms go, you can use either Kubernetes or Docker Swarm for your Docker engines. Kubernetes currently holds the largest market share and is pretty much the standard platform. It works around the concept of pods, which are scheduling units (and can contain one or more containers) in the Kubernetes ecosystem and they are distributed among nodes to provide high availability.

It has the advantage of using years of research done by Google, hence leveraging the expertise. Being open source, it has a vibrant, growing community with plenty of helpful resources, and guidance available to anyone looking. It can run on either a public cloud service or on premises, and is easy to learn and implement.

Docker Swarm has the advantage of tightly integrated into the Docker ecosystem, and uses its own API. Its filtering and scheduling system enables the selection of optimal nodes in a cluster to deploy containers. As it is developed by Docker itself, Docker Swarm removes many compatibility and other differences and integrates smoothly. Kubernetes usually is not a complete solution and requires custom plug-ins to set up.

Sometimes production applications will span across multiple containers, and those containers may be deployed across multiple physical server machines. Both Kubernetes and Swarm give you the orchestration and management capabilities required to deploy and scale those containers to accommodate the always changing workload requirements.

Architecture

Kubernetes follows the master-slave architecture. The components of Kubernetes can be divided into those that manage an individual node and those that are part of the control plane.

Kubernetes control plane (master)

The Kubernetes Master is the main controlling unit of the cluster, managing its workload and directing communication across the system. The Kubernetes control plane consists of various components:

• etcd: Consistent and highly-available key value store used as Kubernetes’ backing store for all cluster data. Practically speaking I was never required to bother how etcd works, its well encapsulated within the control plane boundaries and is efficiently managed by Kubernetes.
• API-Server: Component on the master that exposes the Kubernetes API. It is the front-end for the Kubernetes control plane. Every time we execute a kubectl command, it goes directly to the api server and is executed by it after taking a good look at the configuration and available hardware.
• Kube-Scheduler: Component on the master that watches newly created pods that have no node assigned, and selects a node for them to run on.
• Kube-Controller-Manager: Component on the master that runs controllers. Logically, each controller is a separate process, but to reduce complexity, they are all compiled into a single binary and run in a single process. These controllers include:
• Node Controller: Responsible for noticing and responding when nodes go down.
• Replication Controller: Responsible for maintaining the correct number of pods for every replication controller object in the system.
• Endpoints Controller: Populates the Endpoints object (that is, joins Services & Pods).
• Service Account & Token Controllers: Create default accounts and API access tokens for new namespaces.

Kubernetes node (slave)

The Node, also known as Worker or Minion, is a machine where containers are deployed. If you take example of AWS, you can say these nodes are nothing but simple EC2 instances with a given RAM and CPU configuration.

Every node in the cluster must run a container runtime such as Docker, as well as the below-mentioned components, for communication with master for network configuration of these containers.

• Kubelet: An agent that runs on each node in the cluster. It makes sure that containers are running in a pod. The kubelet takes a set of PodSpecs that are provided through various mechanisms and ensures that the containers described in those PodSpecs are running and healthy.
• Kube-proxy: Enables the Kubernetes service abstraction by maintaining network rules on the host and performing connection forwarding.
• Container runtime: The container runtime is the software that is responsible for running containers. Kubernetes supports several runtimes: Docker, containerd, cri-o, rktlet and any implementation of the Kubernetes CRI (Container Runtime Interface).

Kubernetes Objects

Kubernetes defines a set of building blocks (“primitives”), which collectively provide mechanisms that deploy, maintain, and scale applications based on CPU, memory or custom metrics.

This extensibility is provided in large part by the Kubernetes API, which is used by internal components as well as extensions and containers that run on Kubernetes. The platform exerts its control over compute and storage resources by defining resources as Objects, which can then be managed as such. The key objects are:

• Pods

A Pod is a Kubernetes abstraction that represents a group of one or more application containers (such as Docker or rkt), and some shared resources for those containers such as volumes, DNS etc.

A Pod models an application-specific “logical host” and can contain different application containers which are relatively tightly coupled. For example, a Pod might include both the container with your Node.js app as well as a different container that feeds the data to be published by the Node.js webserver. The containers in a Pod share an IP Address and port space, are always co-located and co-scheduled, and run in a shared context on the same Node.

Example of a Pod definition:

apiVersion: v1
kind: Pod
metadata:
  name: myapp-pod
  labels:
    app: myapp
spec:
  containers:
  - name: myapp-container
    image: busybox
    command: ['sh', '-c', 'echo Hello Kubernetes! && sleep 3600']

• Services

A Kubernetes Service is an abstraction which defines a logical set of Pods and a policy by which to access them - sometimes called a micro-service. The set of Pods targeted by a Service is (usually) determined by a Label Selector.

Kubernetes Pods are mortal. They are born and when they die, they are not resurrected. ReplicaSets in particular create and destroy Pods dynamically. While each Pod gets its own IP address, even those IP addresses cannot be relied upon to be stable over time. This leads to a problem: if some set of Pods (let’s call them backends) provides functionality to other Pods (let’s call them frontends) inside the Kubernetes cluster, how do those frontends find out and keep track of which backends are in that set? This is where the concept of a service kicks in.

Example of a Service definition:

kind: Service
apiVersion: v1
metadata:
  name: my-service
spec:
  selector:
    app: MyApp
  ports:
  - protocol: TCP
    port: 80
    targetPort: 9376

• Volumes

In Docker, a volume is simply a directory on disk or in another Container. Lifetimes are not managed and until very recently there were only local-disk-backed volumes. Docker now provides volume drivers, but the functionality is very limited for now (e.g. as of Docker 1.7 only one volume driver is allowed per Container and there is no way to pass parameters to volumes).

A Kubernetes volume, on the other hand, has an explicit lifetime — the same as the Pod that encloses it. Consequently, a volume outlives any Containers that run within the Pod, and data is preserved across Container restarts. Of course, when a Pod ceases to exist, the volume will cease to exist, too. Perhaps more importantly than this, Kubernetes supports many types of volumes, and a Pod can use any number of them simultaneously.

At its core, a volume is just a directory, possibly with some data in it, which is accessible to the Containers in a Pod. How that directory comes to be, the medium that backs it, and the contents of it are determined by the particular volume type used.

AWS EBS Example configuration:

apiVersion: v1
kind: Pod
metadata:
  name: test-ebs
spec:
  containers:
  - image: k8s.gcr.io/test-webserver
    name: test-container
    volumeMounts:
    - mountPath: /test-ebs
      name: test-volume
  volumes:
  - name: test-volume
    # This AWS EBS volume must already exist.
    awsElasticBlockStore:
      volumeID: <volume-id>
      fsType: ext4

• Namespaces

Kubernetes namespaces can be seen as a logical entity used to represent cluster resources for usage of a particular set of users. This logical entity can also be termed as a virtual cluster. One physical cluster can be represented as a set of multiple such virtual clusters (namespaces). The namespace provides the scope for names. Names of resources within one namespace need to be unique.

By default, Kubernetes starts with the following three namespaces:

• Default: Catch-all namespace for all objects not belonging to either of the kube-public or kube-system namespaces. The default namespace is used to hold the default set of pods, services, and deployments used by the cluster.
• Kube-public: Namespace for resources that are publicly available/readable by all
• Kube-system: Namespace for objects/resources created by Kubernetes systems

kubectl

Kubectl controls the Kubernetes Cluster. It is one of the key components of Kubernetes which runs on the workstation on any machine when the setup is done. It has the capability to manage the nodes in the cluster. Kubectl commands are used to interact and manage Kubernetes objects and the cluster. Some of the important commands are as follows:

• kubectl apply − It has the capability to configure a resource by file or stdin.

$ kubectl apply –f <filename>

• kubectl attach − This attaches things to the running container.

$ kubectl attach <pod> –c <container>
$ kubectl attach 123456-7890 -c tomcat-conatiner

• kubectl autoscale − This is used to auto scale pods which are defined such as Deployment, replica set, Replication Controller.

$ kubectl autoscale (-f FILENAME | TYPE NAME | TYPE/NAME) [--min = MINPODS] --max = MAXPODS [--cpu-percent = CPU] [flags]
$ kubectl autoscale deployment foo --min = 2 --max = 10

• kubectl cluster-info − It displays the cluster Info.

$ kubectl cluster-info

Web UI (Dashboard)

Dashboard is a web-based Kubernetes user interface. You can use Dashboard to deploy containerised applications to a Kubernetes cluster, troubleshoot your containerised application, and manage the cluster resources. You can use Dashboard to get an overview of applications running on your cluster, as well as for creating or modifying individual Kubernetes resources (such as Deployments, Jobs, DaemonSets, etc). For example, you can scale a Deployment, initiate a rolling update, restart a pod or deploy new applications using a deploy wizard.

Dashboard also provides information on the state of Kubernetes resources in your cluster and on any errors that may have occurred. Some sample images from the starter-kit we are going to discuss in part-2 of this series are as follows:

Conclusion

This concludes the first part of the series on Kubernetes. I hope via this illustration, the readers would have received some insight into what this new technology entails.

The second part of the series is a little more hands-on kind which will allow you to setup a Kubernetes cluster of your own in a matter of minutes with all the prominent Kubernetes objects covered in-depth along with practical examples of deploying a microservices based product on it.

If it was interesting or helpful to you, please do press the 👏 clap button and help others find this story too.

Did you know you could give up to 50 claps?

What is Kubernetes? was originally published in The Resonant Web on Medium, where people are continuing the conversation by highlighting and responding to this story.

In continuation to the first part of the series on Spring Boot v2.0, this is the second and concluding part where we will go through the application project structure along with some important snippets and best practices which should be considered while doing Spring Boot based development.

Just a small disclaimer, after setting up this starter-kit and doing a thorough review, you might feel that this is not actually a micro service implementation using Spring Boot, I must say I completely agree with you it is not, rather this is a monolith developed in Spring Boot. This article is more about learning to code with the best practices in Spring Boot and ideally a reservation system should have more than just one service or code base, I promise I will soon be publishing a new series on Spring Boot’s usage in creating a distributed application with the help of Netflix OSS stack. Till then please treat this as a starting point and get the project structure and related tools aligned.

Source Code

The source code for this starter kit with MongoDB as the database can be cloned from the following GitHub repository :

khandelwal-arpit/springboot-starterkit

The MySQL version of the same is available here:

khandelwal-arpit/springboot-starterkit-mysql

Application Structure

Spring Boot is an opinionated framework that makes our life very easy since we don’t have to choose the versions of different dependencies based on the version of Spring framework, its all taken care of by Spring Boot.

I have tried to follow the same ideology while creating the project structure, at first it might seem like overwhelming, but do believe me once you start writing your pieces the structure will help you immensely by saving your time and thinking about questions which are already answered. The structure look as follows :

Models & DTOs

The various models of the application are organised under the model package, their DTOs(data transfer objects) are present under the dto package. There are different opinions about whether we should use DTOs or not, I belong to the set of minds who think we definitely should and not using DTOs makes your model layer very tightly coupled with the UI layer and that is something that no enterprise project should ever get into.

DTOs let us transfer only the data that we need to share with the user interface and not the entire model object that we may have aggregated using several sub-objects and persisted in the database. The mapping of models to the DTOs can be handled using ModelMapper utility, however its only useful when your DTO is almost similar (literally) to the concerned models which is not always the case and hence I prefer using custom mapper classes. You can find some examples under the “dto/mapper” package.

For ex, let’s take a look at how our Trip.java model is organised :

Trip.java

As we can see the model has references to the Stop, Bus and Agency collections. This is required to maintain the relationship between these various collections with a Trip. While this is not the same as a foreign key concept in MySQL and there is no default cascading applied here, it does give us a way to emulate the same in MongoDB. The corresponding data transfer object looks as follows :

TripDto.java

As I described earlier, DTOs are not meant to be the mirror image of their model counterparts, rather they should be a reflection of what the user interface or the api response demands. In this case there was no sense in making a composition relationship between a Trip and its Bus or Agency or the Stops, rather their primary keys actually can do the trick. This not only decouples these DTOs, it also reduces the overall size of the response packet that will travel over HTTP from server to the client.

The very next question can be how to convert the model POJO to this DTO, well there are more than one ways to do it but my preference is to be explicit and DIY as follows :

Services and DAOs

The data access objects (DAOs) are present in the repository package. They are all extensions of the MongoRepository interface helping the service layer to persist and retrieve the data from MongoDB.

The service layer is defined under the service package, considering the current case study it made sense to create two basic services :

1. UserService and
2. BusReservationService

Explaining each method in these services is out of the scope of this blog post, however I will list out the interfaces showing the operations supported by these services.

UserService.java

BusReservationService.java

Apart from noticing the method naming conventions, I am sure you have noticed that the service layer never accepts a model as input and never ever returns one either. This is another best practice that Spring developers should follow in a layered architecture. The controller layer interacts with the service layer to get a job done whenever it receives a request from the view or api layer, when it does it should not have access to the model objects and should always converse in terms of neutral DTOs.

When the service receives a DTO object, it should try to make sense out of it by querying for the corresponding model object from database and then do the necessary operation and create a response DTO to be sent back to the calling service. This approach lets you change the view and models independently without having to worry one might break the other.

To illustrate the aforementioned approach, let’s take an example of a service operation “updateProfile” that is used to update user’s information. The method definition look as follows :

Security

The security setting are present under the config package and the actual configurations are done under the class present in the security package. The application has different security concepts for the admin portal and the REST APIs, for the portal I have applied the default spring session mechanism that is based on the concept of sessionID and cookies. For the REST APIs I have used JWT token based authentication mechanism.

The security for web and apis are configured both in the same class MultiHttpSecurityConfig.java. It has two static classes extending from WebSecurityConfigurerAdapter which let’s us configure the http security for incoming requests.

The @Order annotation allows the requests to be scanned through the different configurations in the specified order. So an API request goes through the ApiWebSecurityConfigurationAdapter and gets absorbed there, however an Admin request first goes through it but since it doesn’t match the criteria Spring Security tries to make it go through the next configuration with an immediate higher order which in this case is FormLoginWebSecurityConfigurerAdapter.

The API requests have to go through the ApiJWTAuthenticationFilter and ApiJWTAuthorizationFilter which are responsible for the creation and validation of the JWT token issued at the time of login. If you are wondering which URL should be used for API authentication (login) then here it is :

http://localhost:8080/api/auth

And if you are wondering how this was configured, then the answer lies in the class ApiJWTAuthenticationFilter, its constructor has the following information coded :

this.setRequiresAuthenticationRequestMatcher(new AntPathRequestMatcher("/api/auth", "POST"));

This tells AbstractAuthenticationProcessingFilter to hook the AuthenticationRequestMatcher with “/api/auth” path for the API requests.

The admin application is only allowed for those users whose role is “ADMIN”. All the passwords are encrypted using BCryptPasswordEncoder and we can never see their actual value once they are stored in the database.

Controllers

Last, but the most important part is the controller layer. It binds everything together right from the moment a request is intercepted till the response is prepared and sent back. The controller layer is present in the controller package, the best practices suggest that we keep this layer versioned to support multiple versions of the application and the same practice is applied here.

For now code is only present under v1 but over the time I expect to have different versions having different features. The Admin portal related controllers are present in the ui package and its concerning form command objects are located under the command package. The REST API controllers are located under the api package and the corresponding request classes are located under the request package.

The admin panel controllers work on the Spring WebMVC concept. They respond to incoming web requests with Spring’s ModelAndView objects containing the data to be shown on the concerned views/forms and the name of the view to be rendered, an example from DashboardController class is as follows :

Request and Form Commands

Again, there are different opinions amongst the developer community regarding the usage of separate classes for mapping the incoming request vs using the DTOs vs using the models directly, I am personally a fan of segregating them as far as possible to promote loose coupling amongst UI and controller layer.

The request objects and the form commands do give us a way to apply an additional level of validations on the incoming requests before they get converted to the DTOs which transfer valid information to the service layer for persistence and data retrieval. We could use DTOs here and some developers prefer that approach as it reduces some additional classes, however I usually prefer to keep the validation logic separate from the transfer objects and hence am inclined to use the request/command objects ahead of them.

An example command pattern based class BusFormCommand looks as follows :

And an example of a request sent via API, BookTicketRequest is as follows :

The static resources are grouped under the resources directory. All the UI objects and their styling aspects can be located here.

Lombok

One of the biggest complaints against Java is how much noise can be found in a single class. Project Lombok saw this as a problem and aims to reduce the noise of some of the worst offenders by replacing them with a simple set of annotations. You will find Lombok employed everywhere in this starter kit, it has actually helped in reducing the lines of code, save a lot of development time and effort and make the code a lot more readable. Some of the most important annotations that I prefer using are :

@Getter /@Setter

Never write public int getFoo() {return foo;} again.

@ToString

No need to start a debugger to see your fields: Just let lombok generate a toString for you!

@EqualsAndHashCode

Equality made easy: Generates hashCode and equals implementations from the fields of your object..

@Data

All together now: A shortcut for @ToString, @EqualsAndHashCode, @Getter on all fields, and @Setter on all non-final fields, and

@RequiredArgsConstructor!

In essence, something as verbose as :

Could be written simply as :

You can very well see the difference, not only the later looks much cleaner, we have come down from 59 lines of boring POJO to a 8 lines based java class file.

API Response and Exception Handling

I have tried to experiment a bit with the RuntimeExceptions and come up with a mini framework for handling the entire application’s exceptions using a few classes and the properties file. If you carefully observe the exception package, it consists of two enums :

1. EntityType and
2. ExceptionType

The EntityType enum contains all the entity names that we are using in the system for persistence and those which can result in a run time exception. The ExceptionType enum consists of the different entity level exceptions such as the EntityNotFound and DuplicateEntity exceptions.

The BRSException class has two static classes EntityNotFoundException and DuplicateEntityException which are the two most widely thrown exceptions from the service layer. It also contains a set of overloaded methods throwException which take the EntityType, ExceptionType and arguments to come up with a formatted message whose template is present under the custom.properties file.

Using this class I was able to empower the entire services layer to throw a variety of entity exceptions in a uniform manner without cluttering the code base with all sorts of NOT_FOUND and DUPLICATE entity exceptions.

For example, while login if you try to use a email address which is not registered, an exception is raised and thrown using the following single line of code -

throw exception(USER, ENTITY_NOT_FOUND, userDto.getEmail());

This results in clubbing the names of these two enum elements USER(“user”) and ENTITY_NOT_FOUND(“not.found”) and coming up with a key user.not.found which is present in the custom.properties files as follows :

user.not.found=Requested user with email - {0} does not exist.

By simply replacing the {0} param with the email address included in the thrown exception you can get a well formatted message to be shown to the user or to be sent back as the response of the REST API call. The definition of the BRSException class is as follows :

Another important part of this mini framework is the CustomizedResponseEntityExceptionHandler class which looks as follows :

This class takes care of these RuntimeExceptions before sending a response to the API requests. Its a controller advice that checks if a service layer invocation resulted in a EntityNotFoundException or a DuplicateEntityException and sends an appropriate response to the caller.

Last, the API response are all being sent in a uniform manner using the Response class present in the dto/response package. This class allows us to create uniform objects which result in a response as shown below (this one is a response for the “api/v1/reservation/stops” api) :

{
 “status”: “OK”,
 “payload”: [
   {
    “code”: “STPA”,
    “name”: “Stop A”,
    “detail”: “Near hills”
   },
   {
    “code”: “STPB”,
    “name”: “Stop B”,
    “detail”: “Near river”
   }
 ]
}

And when there is an exception (for example searching for a trip between two stops which are not linked by any bus) the following responses are sent back (result of “api/v1/reservation/tripsbystops” GET request) :

{
  “status”: “NOT_FOUND”,
  “errors”: “No trips between source stop — ‘STPD’ and destination  stop — ‘STPC’ are available at this time.”
}

{
  “status”: “NOT_FOUND”,
  “errors”: 
  {
    “timestamp”: “2019–03–13T07:47:10.990+0000”,
    “message”: “Requested stop with code — STPF does not exist.”
  }
}

As you can observe, both type of responses, one with a HTTP-200 and another with HTTP-404 the response payload doesn’t change its structure and the calling framework can blindly accept the response knowing that there is a status and a error or payload field in the JSON object.

API Documentation

It is as important to document your work (as is its development) and make sure your Spring Boot APIs are available in a readable manner for frontend teams(internal) or external consumers. The tool for API documentation used in this starter kit is Swagger2, you can open the same inside a browser at the following url -

http://localhost:8080/swagger-ui.html

It will present you with a well structured UI which has two specs :

1. User
2. BRS

You can use the User spec to execute the login api for generating the Bearer token. The token then should be applied in the “Authorize” popup which will by default apply it to all secured apis (get and post both). Please bear in mind that you should append the word “Bearer” followed by a space, in front of the token before applying it in the authorize dialog box.

The configuration of Swagger is being taken care of by class BrsConfiguration. I have defined two specs there with the help of “swaggerBRSApi” and “swaggerUserApi” methods. Since the login part is by default taken care of by Spring Security we don’t get to expose its apis implicitly as the rest of the apis defined in the system and for the same reason I have defined a controller in the config package with the name “FakeController” :

Its purpose is to facilitate the generation of swagger documentation for login and logout apis, it will never come into existence during the application life cycle as the “/api/auth” api is being handled by the security filters defined in the code base. Here are some sample screenshots to help you visualise things a little better :

To use the Swagger UI and execute the secured APIs, you will need to first operate the “/api/auth” from the User spec and generate a Bearer token. Once the token is issued, you can register it in the Authorize popup and then move on to the BRS spec to execute the secured APIs. If you don’t register the token then you will keep on receiving the HTTP 401 error.

You may get into issues while trying out the Swagger-Ui for those HTTP-GET requests which have a body with parameters, don’t worry it’s not the fault of the code it is Swagger’s decision to stop supporting body params for GET and DELETE requests. As a workaround, you may copy the curl request from swagger and execute it inside a terminal window and that should work fine or you may opt to go for Postman or a similar tool as they don’t (yet) enforce such a tight restriction. In my views, as long as the Open API specifications don’t restrict the body params in GET requests, tools like Swagger shouldn’t too, however its a call that they need to make and not us as developers.

UI Architecture

The user interface for the admin portal is designed using material design with the help of Bootstrap and responsive web app concept. The UI is server side rendered using Thymeleaf templates (preferred templating engine in Spring).

The standard way of working with Thymeleaf is to use includes. This quite often leads to repetitive code, especially, when a website has many pages and each page has several reusable components (e.g. header, navigation, sidebar, and footer). It is repetitive as each content page has to include the same fragments at the same locations. This also has a negative effect when the page layout changes, e.g. when putting the sidebar from the left to the right side.

The decorator pattern used by the Thymeleaf Layout dialect solves these issues. In the context of template engines, the decorator pattern doesn’t work with includes on content pages anymore, but it refers to a common template file. Each page basically only provides the main content and by describing which basic template to use the template engine can build the final page. The content is being decorated with the template file.This approach has advantages compared to the standard way of including fragments:

• The page itself only has to provide the content.
• As a template file is being used to build the final page global changes can be applied easily.
• The code becomes shorter and cleaner.
• As each content page references which template file to use, it is easy to use different templates for different areas of the application (e.g. public area and admin area).

The layout for admin portal is designed as follows :

The individual areas in this layout serve the following purpose :

• Header : this fragment is used for the static imports (CSS and JavaScript), the title and meta tags.
• Navigation : the navigation bar with user profile at top right.
• Content : the content placeholder that will be replaced by the requested page.
• Sidebar : a sidebar for additional information.
• Footer : the footer area that provides the copyright info.

These components can be located in the resources/templates directory at the root as well as under the sub-directories fragments and layout. The content area in this layout will host the following pages :

• Dashboard
• Agency
• Bus
• Trip
• Profile

Also, an error page for any unhandled exception is designed with the name “error.html”. The login and signup pages are designed separately from the portal accessible to a logged-in user.

Running the Server Locally

To be able to run this Spring Boot app you will need to first build it. To build and package the Spring Boot app into a single executable Jar file with a Maven, use the below command. You will need to run it from the project folder which contains the pom.xml file.

maven package

or you can also use

mvn clean install

To run the app from a terminal window you can you the java -jar command. This is provided your Spring Boot app was packaged as an executable jar file.

java -jar target/springboot-starterkit-0.0.1-SNAPSHOT.jar

You can also use Maven plugin to run the app. Use the below example to run your Spring Boot app with Maven plugin :

mvn spring-boot:run

You can follow any/all of the above commands, or simply use the run configuration provided by your favourite IDE and run/debug the app from there for development purposes. Once the server is setup you should be able to access the admin interface at the following URL :

http://localhost:8080

And the REST APIs can be accessed over the following base-path :

http://localhost:8080/api/

Some of the important api endpoints are as follows :

• http://localhost:8080/api/v1/user/signup (HTTP:POST)
• http://localhost:8080/api/auth (HTTP:POST)
• http://localhost:8080/api/v1/reservation/stops (HTTP:GET)
• http://localhost:8080/api/v1/reservation/tripsbystops (HTTP:GET)
• http://localhost:8080/api/v1/reservation/tripschedules (HTTP:GET)
• http://localhost:8080/api/v1/reservation/bookticket (HTTP:POST)

Docker Container

Please use the following command to build the container image :

docker build -t spring/starterkit .

And the following command to run the container :

docker run -p 8080:8080 spring/starterkit

Please note when you build the container image and if mongodb is running locally on your system, you will need to provide your system’s IP address (or cloud hosted database’s IP) in the application.properties file (or env vars) to be able to connect to the database from within the container.

Conclusion

As you can see, the starter-kit is designed to save you hours of coding time by giving you a streamlined, efficient and clean-coded interface and architecture to base your own development on. Try it out and be sure to let me know your feedback.

I hope this series of two articles has helped you in improving your Spring Boot skillset and that it has given you a solid platform to begin writing your next Spring Boot app.

Happy coding everyone!

Did you know you could give up to 50 claps?

If it was interesting or helpful to you, please do press the 👏 clap button and help others find this story too.

If you find this article and associated GitHub project useful, I’d appreciate if you’d buy me a cofee.

Spring Boot 2.0 — Project Structure and Best Practices (Part 2) was originally published in The Resonant Web on Medium, where people are continuing the conversation by highlighting and responding to this story.

The Spring Framework has been around for over a decade and has found a place as the de facto standard framework for developing Java applications. With such a long and storied history, some might think that Spring has settled, resting on its laurels, and is not doing anything new or exciting. Some might even say that Spring is legacy and that it’s time to look elsewhere for innovation.

Some would be wrong.

There are many exciting new things taking place in the Spring ecosystem, including work in the areas of cloud computing, big data, schema-less data persistence, reactive programming, and client-side application development. Perhaps the most exciting, most head-turning, most game-changing new thing to come to Spring in the past year or so is Spring Boot. Spring Boot offers a new paradigm for developing Spring applications with minimal friction.

— Craig Walls

Preface

I got the inspiration to write this article (first in the series of two) when I was trying to goole my way through the internet to find an ideal project structure (boilerplate) for Spring Boot v2 based projects. Even though Spring Initializr does a decent job to let us create a basic pom.xml (similar to package.json for js developers) using a handy graphical user interface, it doesn’t quite create a complete project structure for us that we can leverage and build our application upon.

Also the fact that there are tons of undocumented github contributions made by spring developers and plethora of misleading articles online, I finally decided to study what’s new in Spring Boot v2 and come up with a project structure that is not just a starting point, but an almost complete bus reservation system of its own written completely in Spring Boot and the user interface designed using Thymeleaf.

In these two articles I would try to cover the basics of the application that I have built along with details on the code structure, various artefacts used for its creation along with some important mini frameworks that I built on the fly to support some of the cross cutting concerns.

Spring Boot

I suppose we all are well aware of the strengths and weaknesses of Spring Framework, specially the complex XML configurations which proved to be a nightmare for enterprise grade applications with hundreds of beans and services to support a requirement.

A lot of work has gone into Spring Boot to reduce complexity and dependencies, which largely alleviates our previous reservations. If you live in a Spring ecosystem and are moving to microservices, Spring Boot is now the obvious choice. Spring Boot allows easy set up of standalone Spring-based applications. It’s ideal for pulling up new microservices and easy to deploy. It also makes data access less of a pain due to the hibernate mappings with much less boilerplate code.

Spring Boot makes it easy to create stand-alone, production-grade Spring based Applications that you can “just run”. It takes an opinionated view of the Spring platform and third-party libraries so that we can get started with minimum fuss. Most Spring Boot applications need very little Spring configuration, it has following distinguishing features:

• Easy dependency Management
• Embed Tomcat, Jetty or Undertow directly (no need to deploy WAR files)
• Provide opinionated ‘starter’ dependencies to simplify your build configuration
• Automatically configure Spring and 3rd party libraries whenever possible
• Provide production-ready features such as metrics, health checks and externalised configuration
• Absolutely no code generation and no requirement for XML configuration
• DevTools to auto restart server on code/config updates
• Easy management of profile specific properties

What’s New in Spring Boot v2.0?

Spring Boot 2.0 is the first major revision of Spring Boot since 1.0 was released years ago and it’s also the first GA version of Spring Boot that provides support for Spring Framework 5.0.

• Minimum version of Java required is Java 8.
• It supports Java 9 too.
• Out of the box support for “Reactive Web Programming” with Spring WebFlux.
• Auto-configuration and starter POMs for reactive Spring Data Cassandra, MongoDB, Couchbase, and Redis.
• Finally support for HTTP/2.
• A freshly written Gradle plugin with minimum requirement of Gradle 4.x
• Support for Kotlin 1.2.x
• Out of the box support for InfluxDB.

A complete list of changes along with basic upgrade guide is located here.

Bus Reservation System

This starter kit focuses on how to use Spring Boot, following all the best practices that are recommended by Spring Framework 5.0 and Spring Boot v2.0, ensure the code is loosely coupled and that all the layers in the application are completely independent of each other and talk using neutral objects. While writing this kit, I have done sufficient research around the code structure and usage of appropriate design patterns to make this as an excellent starting point to begin coding your own web application.

The kit would have been incomplete if it did not have a concrete use case to implement, here I have taken a case study of a very simple yet worthy Bus Reservation System and tried to implement an Admin portal which can be operated via browser and a series of REST APIs to interact with the system using mobile applications or frontend applications. The complete systems has two prominent actors :

1. Admin user
2. End user

The Admin user can access this application on browser (laptop or mobile/tablet, doesn’t really matter as this is built using bootstrap, material design and is completely responsive) and can perform the following actions :

1. Signup
2. Login (Spring sessions)
3. Update their profile
4. Create an agency
5. Add buses to the agency
6. Add trips consisting of predefined stops and buses

The End user can use their mobile application (yet to be built, however the REST APIs are ready and could be used via Postman or Swagger) to perform the following actions :

1. Signup
2. Login (and get a JWT token)
3. List all available stops
4. Search a trip between any two stops
5. Filter search results with a date option
6. Book a ticket for a given trip schedule

Admin interface and REST APIs both have their independent authentication mechanisms, the web application uses the cookie based authentication (provided by default by Spring security) and the REST API uses the JWT authentication for access. This application assumes the availability of ‘MongoDB’ installation on the localhost where the server will run or the use of docker-compose to boot up a mongodb container and link the application with it within the realm of docker.

Any changes that the admin users will do on the web portal will impact the search results of the end users, there will be certain use cases which you may find missing here, I hope you will appreciate that the overall idea was to present a way to create such an application completely inside the realm of Spring Boot and not to actually building a fully operational reservation system.

The admin user interface is completely written in material design using Bootstrap v4 and is responsive to suite a variety of devices. The template engine used to render the admin views is Thymeleaf since the library is extremely extensible and its natural templating capability ensures templates can be prototyped without a back-end — which makes development very fast when compared with other popular template engines such as JSP.

Database Schema

The current schema looks as follows:

• The authentication and authorization is governed by User and Role collection.
• The Agency collection keeps the details of agency along with who owns it.
• The Stop collection keeps the data about all the stops in the system.
• The Bus collection has the data of all the buses for various agencies along with their capacity and make years.
• The Trip and TripSchedule collections keep the information about all the trips that agency owners create within the system. Information like source and destination stops, journey time, data of travel, tickets sold so far and the available seats is collected in them.
• Last, the Ticket collection keeps information about all the tickets issued in the application for various trips.

Some of you might wonder the reason behind choosing String data type for the dates instead of Date data type, well, to be honest MongoDB has its own way to deal with dates and tends to use the UTC format which led to some avoidable complications within the codebase and hence the preferred approach was to go with a String instead and do some calculations in house within the java code if needed.

Libraries/Tools Used

Following libraries were used during the development of this starter kit :

• Spring Boot : Server side framework
• Docker : Containerising framework
• MongoDB : NoSQL database
• Swagger : API documentation
• Thymeleaf : Templating engine
• Material : UI themeing/design
• Bootstrap : CSS framework
• JWT : Authentication mechanism for REST APIs

User Interface

Here are the various screens of the Admin portal that you should be able to use once the application is setup properly :

Login

Signup

Dashboard

Agency

Buses

Trips

Profile

Part 2

Thanks for reading the first part from a series of two articles on Spring Boot. If this is enough to raise your interest in using this impressive web framework, be sure to check out the second part here.

It contains a detailed walkthrough of the starter-kit app built by me, which provides an excellent starting point for all Spring Boot developers to use in their own production environments. It will cover topics such as the project structure, dockerisation, exception handling, web security, decorated Thymeleaf view layout etc.

Did you know you could give up to 50 claps?

If it was interesting or helpful to you, please do press the 👏 clap button and help others find this story too.

If you find this article and associated GitHub project useful, I’d appreciate if you’d buy me a cofee.

Spring Boot 2.0 — Project Structure and Best Practices (Part 1) was originally published in The Resonant Web on Medium, where people are continuing the conversation by highlighting and responding to this story.

I recently came across a question from a peer in the organisation — “what are the top web development stacks and how do we decide the right stack for the problem in hand?” I wanted to redirect the question towards internet, so I started looking for a decent guide that could help in answering this question. Unfortunately, I couldn’t come across a single writeup that answered the different facets related to this question. This was the motivation behind coming up with a definitive guide here that lists some of the key concepts of web development and the tools and technologies available today to achieve the goal of a scalable web application.

Before you start reading any further, please bear in mind this article is not about a list of ‘hottest languages or tech stacks’ for web application development throwing a cluster of buzzwords at you, rather this is a practical review of the trends in the industry backed up by statistics from various online sources and in-depth research mixed with years of experience working in the field of web applications.

According to wikipedia, “Web development is a broad term for the work involved in developing a website for the Internet (World Wide Web) or an intranet (a private network). Web development can range from developing the simplest static single page of plain text to the most complex web-based internet applications (or just ‘web apps’) electronic businesses, and social network services. A more comprehensive list of tasks to which web development commonly refers, may include web engineering, web design, web content development, client liaison, client-side/server-side scripting, web server and network security configuration, and e-commerce development. Among web professionals, “web development” usually refers to the main non-design aspects of building web sites: writing markup and coding.”

While I agree with the entire description provided by wikipedia, the most important part to me was the closing line in the above para. As a web professional the first thing that we need to understand is the composition of a typical web application –

Front-end

This is the interface that a developer creates for interaction with the main application. This can be as simple as a static web page designed in HTML or a complex implementation done using a technology based on virtual dom (such as React.js) for browsers or a mobile application developed natively using android and iOS technologies or a hybrid platform such as React Native.

Back-end

This refers to the server side master program running on single or distributed topology and responsible for catering to the stateful and stateless requests sent from the front-end. There is a huge spectrum of technologies available currently to write the back-end implementation of a web application, we will be reviewing some of those in the sections to come.

Datastore

This is the organised collection of information that the front-end usually demands on from the back-end. The datastore (a.k.a. database) contains every persistent information that is useful in the present date as well in future to come up with a custom metrics based on the data gathered over a given period of time. There are two primary divisions in terms of the ways to store the data and those are-

• SQL database and
• NoSQL database

Cloud Hosting Service

This is the information technology (IT) paradigm that enables ubiquitous access to shared pools of configurable system resources and higher-level services that can be rapidly provisioned with minimal management effort, often over the Internet. Cloud computing relies on sharing of resources to achieve coherence and economies of scale, similar to a public utility. The idea in general is to take the complexities of setting decent hardware for deploying the web applications to another level by abstracting the details from routine back-end dev and letting domain experts like AWS and Azure take care of these. This not just reduces the overall cost, it also makes the application secure in a lot of sense by applying default security configurations while deploying the applications. The time it takes to spin a new server on cloud hosting services is close to minutes as compared to hours, if you do the same task in house.

Front-end Technologies

“A front-end dev is responsible for the interior design of a house that’s been built by a back-end dev.”
                                                            -twitter

As mentioned earlier, the front-end of a web application is the part that end users interact with to gain information of the system or to share their thoughts.

There are two promising technologies that I would like to present to the reader in the context of front-end development in current scenario –

React.js

React.js is a front-end library based on JavaScript and built and maintained by Facebook. It abstracts away the DOM from you, giving a simpler programming model and better performance. React can also render on the server using Node.js, and it can power native mobile apps using React Native. The core objective of this library is providing the best possible rendering performance and it draws its strength from individual components instead of focusing on the entire web app at once.

Even though Angular has been in existence for a substantial time, it never came across as something that is easy to use and maintain and the fact that it has gone through major revisions in last two years has rendered the initial knowledge on the technology pretty much useless. Also, Google (founders of Angular) didn’t do a lot of justice with Angular by using closure-libraryinstead of promoting Angular in their top notch public facing applications (including but not limited to Gmail). React on the other hand is not only maintained by Facebook, also it is used heavily by them. PayPal, Uber, Instagram, and Airbnb are just some names in the list of React admirers along with close to 95,000 live websites based on React.

Vue.js

Vue (pronounced /vjuː/, like view) is a progressive framework for building user interfaces. Unlike other monolithic frameworks, Vue is designed from the ground up to be incrementally adoptable.

The core library is focused on the view layer only, and is easy to pick up and integrate with other libraries or existing projects. On the other hand, Vue is also perfectly capable of powering sophisticated Single-Page Applications when used in combination with modern tooling and supporting libraries.

Vue.js is one of those new software technologies that is being widely used across the world for web development. Vue.js is actually a JavaScript framework with various optional tools for building user interfaces. There are numerous advantages of using Vue in web frontend development –

• It’s small size
  The size of this framework is 18–21KB and it takes no time for the user to download and use it. This does not mean that it has low speed because of small size. Instead, it beats all the bulky frameworks like React.js, Angular.js, and Ember.js.
• It’s simply easy
  Personally I am a big fan of Vue, even without a proper front-end development experience I was able to learn and adapt Vue in daily development tasks in less than two weeks.
• Vue’s documentation is admirable
  Even though it is not backed up by multi billion dollar organisations such as Google or FB, it still has managed to evolve in a very short span of time and has ensured that the updates are not as problematic to absorb as has been the case with some of its competitors.
• Offers a great deal of flexibility
  It allows the user to write their template in HTML file, JavaScript file, and pure JavaScript file using virtual nodes. This flexibility also makes it easy to understand for React, Angular or any other JS developers.

Vue or React?

Here is a result of a recent survey that was conducted online to assess the percentage of front-end developers who would like to reuse their past frameworks/libraries for UI development.

No surprises, the top two contenders are React and Vue. This itself speaks a lot about the comfort and faith that these two technologies have now gained over their competitors. Another important consideration is the speed of DOM rendering. If we were to consider vanilla JavaScript as the fastest mode of component rendering, Vue 2.0 is only 1.37 times slower as compared to 1.82 times slow React 15.3 and 1.85 times slower Angular 2.0.

If you are still looking for some more guidelines on choosing one over the other, here are a few important points to consider –

Vue has the following advantages over React:

• it is smaller and faster
• convenient templates simplify the process of developing
• it is simpler in terms of syntax
• 92,500+ Github Stars and 189 contributors

React has the following advantages over Vue:

• gives more flexibility in large apps developing
• easier to test
• suitable for mobile apps creating
• more information and solutions available
• 94,500+ Github Stars and 1177 contributors

Back-end Technologies
The back-end stack, comprises of three important layers –

• Server-side software (application server/web server)
• A set of APIs to collect and serve data
• Database to store and return data as requested by APIs

The server is a powerful hardware (physical or virtual) that runs in the back-end software and database. It’s primary job is to serve or collect data via Application Programming Interfaces (a.k.a. API). The front-end applications, be it browser based thick clients or native/hybrid mobile applications, connects with the server by invoking one of the many live APIs on the server and GET or POST the data which is eventually stored in the database. APIs play an integral role in how most server-side software architectures are built, oftentimes replacing more complicated programming to allow software to communicate and data to be transferred.

The intricacies of the back-end stack are beyond the scope of this article and the next section focuses on some of the most promising server side technologies/frameworks which are available as options to the back-end developer today –

Node.js

As per Wiki : Node.js is an open-source, cross-platform JavaScript run-time environment for executing JavaScript code server-side. Historically, JavaScript was used primarily for client-side scripting, in which scripts written in JavaScript are embedded in a webpage’s HTML, to be run client-side by a JavaScript engine in the user’s web browser.

Node.js was first conceived in 2009 by Ryan Dahl and he continued to develop and maintain it till it got sponsored and supported by Joyent. Dahl was not happy with the way Apache Http server used to handle the lot of concurrent connections and the way code was being created which either blocked the entire process or implied multiple execution stacks in the case of simultaneous connections. This lead him to create a Node.js project, he used Google’s V8 JavaScript engine, an event loop, and a low-level I/O API in his project which won him a standing ovation.

Node JS Platform does not follow Request/Response Multi-Threaded Stateless Model. It follows Single Threaded with Event Loop Model. Node JS Processing model is mainly based on Javascript Event based model with Javascript callback mechanism. Node.js comprises of two main component core & its modules. Core: built in C & C++. It combines Google V8 JS engine with Node’s Libuv library and protocol bindings including sockets & Http.

Node.js is especially good for developing real-time apps intended for simultaneous connection of multiple users. However, it is not recommended for apps intended for complex computations. Some of the big names you know — Netflix, Trello, PayPal, LinkedIn, Walmart, Uber, Groupon, eBay, NASA — all use Node.js for their apps.

Some of the typical use cases of Node.js can be listed as follows –

• I/O bound Applications
• Data Streaming Applications
• Data Intensive Real-time Applications (DIRT)
• JSON APIs based Applications
• Single Page Applications

Some of the most promising frameworks built on top of Node.js are –

Python

As per wiki: Python is an interpreted high-level programming language for general-purpose programming. It was created by Guido van Rossum and first released in 1991. Python features a dynamic type system and automatic memory management. It supports multiple programming paradigms, including object-oriented, imperative, functional and procedural, and has a large and comprehensive standard library.

Python is one of those rare languages which can claim to be both simple and powerful. You will find yourself pleasantly surprised to see how easy it is to concentrate on the solution rather than the syntax and structure of the language. Websites built in Python include the likes of Google, Yahoo Maps, Dropbox, Venmo, YouTube, Instagram, Spotify, Reddit, and BitTorrent. Blender 3D, Unity 3D, and Mercurial are some of the well known softwares written in Python. Some of the important features of Python language are –

• Its is a simple and minimalistic language
  Reading a good Python program feels almost like reading English, although very strict English! This pseudo-code nature of Python is one of its greatest strengths. It allows you to concentrate on the solution to the problem rather than the language itself.
• Python is an example of a FLOSS (Free/Libré and Open Source Software)
  In simple terms, you can freely distribute copies of this software, read its source code, make changes to it, and use pieces of it in new free programs.
• It’s a high level language
  When you write programs in Python, you never need to bother about the low-level details such as managing the memory used by your program, etc.
• Python supports procedure-oriented programming as well as object-oriented programming.

Typical use cases for using Python language –

• APIs based development
• Data-heavy sites, servers with high-traffic volume
• Academic and scientific programming. Python has a slew of libraries for data work.
• Machine learning
• Data scientists use Python for prototypes and data analysis

Server side Frameworks built on Python –

Java

Java is a general-purpose computer-programming language that is concurrent, class-based, object-oriented, and specifically designed to have as few implementation dependencies as possible. There were five primary goals while creating Java:

1. It must be “simple, object-oriented, and familiar”.
2. It must be “robust and secure”.
3. It must be “architecture-neutral and portable”.
4. It must execute with “high performance”.
5. It must be “interpreted, threaded, and dynamic”.

The key factor contributing to Java’s success lies in the output of a Java compiler, which is not a executable code. Rather, it is bytecode. Bytecode is a highly optimized set of instructions designed to be executed by the Java run-time system, which is called the Java Virtual Machine (JVM). While Java serves as a great client side programming language, it also is one of the most popular server side choice of language and does power some of the biggest corporate banks and industries such as aviation and communication.

Typical use cases for Java language usage is as follows-

• Multi threaded systems
• Heavy computation
• Machine learning
• Transaction based systems
• Real time communication

Even though J2EE has been in existence for roughly the same amount of time as Java, the Servlet and JSP concepts have remained fairly stable over these years. There have been numerous server side frameworks (MVC and non MVC) written on top of the Servlet specifications given by J2EE, most dominant and prevalent amongst them being Spring.

Most widely used server side frameworks based on Java –

Ruby

Ruby is a language of careful balance. Its creator, Yukihiro “Matz” Matsumoto, blended parts of his favorite languages (Perl, Smalltalk, Eiffel, Ada, and Lisp) to form a new language that balanced functional programming with imperative programming. Since its public release in 1995, Ruby has drawn devoted coders worldwide. In 2006, Ruby achieved mass acceptance with active user groups in the world’s major cities and Ruby-related conferences filled to capacity.

As a dynamically typed language, Ruby does not have hard rules on how to build features, and it is very close to spoken languages. There’s more flexibility in solving problems using different methods. Furthermore, Ruby is also more forgiving of errors, so you’ll still be able to compile and run your program until you hit the problematic part. The emergence of Node.js in recent times has led to a seriously narrower Ruby (and RoR) developer community however it still is used widely for websites such as Hulu, Twitter, ZenDesk, Basecamp, Shopify, Urban Dictionary and GitHub.

It’s almost impossible to talk about Ruby without talking about Rails. In some ways, Rails has become Ruby. Part of this is because of the sheer popularity of the Rails framework. It’s a very popular framework that’s gotten a lot of use and Ruby has come to exist primarily as a service to the Rails framework. It is strongly opinionated, favoring convention over configuration which means unlike Ruby which believes there are many ways to do things, Rails thinks very strongly as far as web development is concerned that there is a best Rails way to do things. It allows you to fast prototyping and is popular with startups wanting to iterate quickly.

Typical use cases for Ruby –

• Web development
• Functional programming
• Getting complex, high-traffic sites and apps up and running quickly

Most widely used server side frameworks written in Ruby –

Elixir

Elixir is a dynamic, functional language designed for building scalable and maintainable applications. Elixir leverages the Erlang VM, known for running low-latency, distributed and fault-tolerant systems, while also being successfully used in web development and the embedded software domain.

Erlang was made at Ericsson in the 1980s to allow for better development of telephony applications. If you use 4G data on your phone it is more likely than not that the data is going through equipment from Ericsson. And it runs Erlang. Whatsapp has hundreds of millions of users. On most other platforms, handling more than ten thousand concurrent connections on one machine is seen as challenging, but Whatsapp has individual servers with more than 2 million concurrent connections, handled by Erlang. Erlang is also popular for game servers. Millions of users play multiplayer games with game server infrastructure handled by Erlang for titles such as Call of Duty and Game of War.

With all these pros, Erlang suffers from some major drawbacks as well. It is not as easy to get up and running a new website, the Erlang syntax looks a bit strange if you are used to looking at languages such as C, Java, JavaScript, PHP, Ruby, Perl etc. And things like package management, build tools, meta-programming, unicode handling and web frameworks are not as straightforward as in languages such as Ruby.

These areas are all addressed in Elixir. Elixir creator José Valim said: “I liked everything I saw in Erlang, but I hated the things that I didn’t see”. Elixir has its own package management system, macros, an easy to use build tool and unicode handling. The difference between Erlang and Elixir is not just the syntax. But syntax does matter, and especially for Ruby developers the Elixir syntax is very familiar.

Typical use cases for using Elixir –

• Writing highly concurrent web applications
• A text chat server that can handle 100–1000000 simultaneous connections per computer
• A telecommunications gateway able to support 1000’s of phones
• Software licensing and inventory system
• Wireshark export parsers
• Connecting IoT devices

Most widely used frameworks written in Elixir –

Hopefully the introductory explanation of these server side technologies will give you a head start towards thinking in terms of back-end development. The choice of either of these technologies for web application development purely depends on the kind of experience you as a programmer have and if you are an entrepreneur, then it depends to an extent on the capabilities of your existing team and their comfort with object oriented vs procedure oriented languages.

At times, it’s a little difficult to join the league of functional programming specially if you started your career as an object oriented language fanatic, however while evaluating certain technology or programming methodology we should always take the factors such as ease of writing code, being opinionated, time to take application live on internet etc in mind. Even if it takes a few extra days or weeks to learn a new language, to me it makes all the sense as it saves a lot more effort in the future when the application goes online.

Along with the choice of server side technology, another very important part of web application development these days is choice of cloud for deployment of the application. There are many available options in the tech industry today (such as but not limited to)-

• AWS
• Azure
• Google Cloud
• Orale Cloud
• Digital Ocean

The choice amongst these boils down to the following factors –

• Cost of service
• Certifications & Standards
• Technologies & Service Roadmap
• Data Security, Data Governance and Business policies
• Service Dependencies & Partnerships
• Contracts, Commercials & SLAs
• Reliability & Performance
• Migration Support, Vendor Lock in & Exit Planning
• Business health & Company profile

While cost should never be the single or most important factor, there’s no denying that cost will play a big role in deciding which cloud service provider you choose. In my personal experience AWS has been the best choice in terms of all the factors mentioned above as well as the cost of individual services. It has reliable data centers across the globe and it keeps on adding new and relevant cloud hosting solutions every year, to me AWS has been an industry leader and a trend setter and others are mostly following everything that AWS comes up with.

With these thoughts, I would like to conclude this article hoping that it was a value addition to new web application developers as well as to the experienced campaigners. This is just the first article in the series of articles to come detailing different aspects of technologies mentioned here. Thanks for reading it through and looking forward to your views.

If it was interesting or helpful to you, please do press the 👏 clap button and help others find this story too.

Did you know you could give up to 50 claps?

How to choose the right technologies for web applications? was originally published in The Resonant Web on Medium, where people are continuing the conversation by highlighting and responding to this story.