Cloud☁️14:Quick & Easy: 3-Node Docker Swarm Environment & Cluster

This is a article that touches base on Docker and Docker Swarm. Let dive in!

What is Docker? — Docker is a software platform that allows you to build, test, and deploy applications quickly. Docker packages software into standardized units called containers that have everything the software needs to run including libraries, system tools, code, and runtime.

What is Docker Swarm? — Swarm mode is an advanced feature for managing a cluster of Docker daemons. Use Swarm mode if you intend to use Swarm as a production runtime environment.

Prequisites:

• Docker installed
• VScode
• AWS Console
• instructions are geared towards MAC

1. Create your nodes for our Docker Swarm Cluster

We will create ec2 instances and rename them nodes.

What is a node? — A node is usually a physical computer system with a distinct host IP address that is running one or more application servers.

What is a cluster? — Clusters are groups of servers that are managed together and participate in workload management. A cluster can contain nodes or individual application servers.

A cluster is one or more nodes grouped together with a manager or main node controlling other nodes

Go too AWS Console > Create Instance >

Create 3 instances (nodes)

Choose name: node1 —

AMI: Ubuntu Server 22.04 LTS.

Free Tier

default

2. Setup Security Groups

• Here we will create a new key pair name: Docker-swarm-key. Ubuntu is Cap sensitive. Once it is downloaded in your MAC, keep it in a safe spot where your laptop can detect the key. I usually place it in the Desktop.

Network Settings:

Create Security Group

Allow SSH: My IP

Go to the first Instance you created and edit the newly created security group called launch-wizard-1.

Click newly created Instance called node1 > Security Groups > click security group > Inbound rules >

Your rules should look like the folllowing:

These ports are needed by docker to work. We can verify this by running netstat -tuplen on the nodes to see what ports are being used.

• 2377 — This port is used for communication between the nodes of a Docker Swarm or cluster. It only needs to be opened on manager nodes.
• 7946 — TCP/UDP for overlay network node discovery
• 2376 — TCP port 2376 for secure Docker client communication. This port is required for Docker Machine to work
• 4789 — Port 4789 is the default value for the Swarm data path port, also known as the VXLAN port. It is important to prevent any untrusted traffic from reaching this port, as VXLAN does not provide authentication. This port should only be opened to a trusted network, and never at a perimeter firewall.
• ALL Traffic to 0.0.0.0/0 — IP address 0.0.0.0 is used on servers to designate a service may bind to all network interfaces. It tells a server to “listen” for and accept connections from any IP address

Port 2377 should be routed to your IP address

What is a CIDR Block? — is a collection of IP addresses that share the same network prefix and number of bits

Once you created your (3) nodes, you should see them running.

nodes that are stopped for charging purposes.

• be sure to set your other nodes to security group 1 launch-wizard-1.

3. Connect your Node Cluster

• Head to VSCode
• We will use the following config file by heading to Users > Sheena > Desktop > config

# Read more about SSH config files: https://linux.die.net/man/5/ssh_config
Host DockerK8s-snarciza
    HostName ec2-54-200-136-83.us-west-2.compute.amazonaws.com
    User ubuntu
    IdentityFile ~/Desktop/Docker-swarm-keypair.pem

• NOTE: Be sure that your Host is the correct name, Hostname, User, and Identity File. Sometimes your Hostname will change if you terminate your node1/ main node.
• We can start connecting to host by clicking the bottom left corner icon.
  See below.

Error — Not connecting to node1?

• When in doubt, restart over! (This helped rather than moving forward and keep trying to fix the remote connection)
• Are you on root?
• Are you connected to node1 from terminal?

ubuntu@ip-172-31-21-70:~$ sudo su
root@ip-172-31-21-70:/home/ubuntu# 

#install dockers
ubuntu@node1:~$ sudo su
root@node1:/home/ubuntu# sudo hostnamectl set-hostname node1
root@node1:/home/ubuntu# install docker

#change directory to root
root@node1:/home/ubuntu# cd /root

#Edit Keypair using VIM
root@node1:~# vim Docker-swarm-keypair.pem

sudo apt update
sudo apt install docker.io

or 

curl -fsSL https://get.docker.com -o get-docker.sh
sh get-docker.sh

Change host name

ubuntu@ip-172-31-0-146:~$ 
hostnamectl set-hostname node1
#OUTPUT:polkit-agent-helper-1: pam_authenticate failed: Authentication failure
#Can't change hostname?

cat > /etc/polkit-1/rules.d/10-some-daemon.rules << POLKIT
polkit.addRule(function(action, subject) {
    if (action.id == "org.freedesktop.systemd1.manage-units" &&
        action.lookup("unit") == "some-daemon.service" &&
        subject.user == "bob") {
        return polkit.Result.YES;
    }
});
POLKIT

sudo reboot

• Your instance will restart/reconnect
• once it reboots, BE SURE TO update your configuration file with the HostName again (this will be your IP address). The IP address/DNS changes each time your EC2 instance is restarted.

Install Docker on your instances:

curl -fsSL https://get.docker.com -o get-docker.sh
sh get-docker.sh

#note if you use other installer it might effect your connection with your cluster

ubuntu@node1:~$ curl -fsSL https://get.docker.com -o get-docker.sh
ubuntu@node1:~$ sh get-docker.sh
# Executing docker install script, commit: c2de0811708b6d9015ed1a2c80f02c9b70c8ce7b
+ sudo -E sh -c apt-get update -qq >/dev/null
+ sudo -E sh -c DEBIAN_FRONTEND=noninteractive apt-get install -y -qq apt-transport-https ca-certificates curl >/dev/null
+ sudo -E sh -c install -m 0755 -d /etc/apt/keyrings
+ sudo -E sh -c curl -fsSL "https://download.docker.com/linux/ubuntu/gpg" | gpg --dearmor --yes -o /etc/apt/keyrings/docker.gpg
+ sudo -E sh -c chmod a+r /etc/apt/keyrings/docker.gpg
+ sudo -E sh -c echo "deb [arch=amd64 signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu jammy stable" > /etc/apt/sources.list.d/docker.list
+ sudo -E sh -c apt-get update -qq >/dev/null
+ sudo -E sh -c DEBIAN_FRONTEND=noninteractive apt-get install -y -qq docker-ce docker-ce-cli containerd.io docker-compose-plugin docker-ce-rootless-extras docker-buildx-plugin >/dev/null
+ sudo -E sh -c docker version
Client: Docker Engine - Community
 Version:           24.0.4
 API version:       1.43
 Go version:        go1.20.5
 Git commit:        3713ee1
 Built:             Fri Jul  7 14:50:55 2023
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          24.0.4
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.20.5
  Git commit:       4ffc614
  Built:            Fri Jul  7 14:50:55 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.21
  GitCommit:        3dce8eb055cbb6872793272b4f20ed16117344f8
 runc:
  Version:          1.1.7
  GitCommit:        v1.1.7-0-g860f061
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

================================================================================

To run Docker as a non-privileged user, consider setting up the
Docker daemon in rootless mode for your user:

    dockerd-rootless-setuptool.sh install

Visit https://docs.docker.com/go/rootless/ to learn about rootless mode.


To run the Docker daemon as a fully privileged service, but granting non-root
users access, refer to https://docs.docker.com/go/daemon-access/

WARNING: Access to the remote API on a privileged Docker daemon is equivalent
         to root access on the host. Refer to the 'Docker daemon attack surface'
         documentation for details: https://docs.docker.com/go/attack-surface/

================================================================================

ubuntu@node1:~$ 

NOTE: It will need to be installed for each instance/node

Get Sheena Lynn Narciza’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

Verify running

ubuntu@node1:~$ sudo docker run hello-world
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
719385e32844: Pull complete 
Digest: sha256:926fac19d22aa2d60f1a276b66a20eb765fbeea2db5dbdaafeb456ad8ce81598
Status: Downloaded newer image for hello-world:latest

Hello from Docker!
This message shows that your installation appears to be working correctly.

To generate this message, Docker took the following steps:
 1. The Docker client contacted the Docker daemon.
 2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
    (amd64)
 3. The Docker daemon created a new container from that image which runs the
    executable that produces the output you are currently reading.
 4. The Docker daemon streamed that output to the Docker client, which sent it
    to your terminal.

To try something more ambitious, you can run an Ubuntu container with:
 $ docker run -it ubuntu bash

Share images, automate workflows, and more with a free Docker ID:
 https://hub.docker.com/

For more examples and ideas, visit:
 https://docs.docker.com/get-started/

Go to root for permission

ubuntu@node1:~$ sudo su
root@node1:/home/ubuntu#

#if cd /root doesn't work use sudo su
root@node1:/home/ubuntu# cd .ssh
root@node1:/home/ubuntu/.ssh# ls
authorized_keys  known_hosts
root@node1:/home/ubuntu/.ssh# 

Create a keypair for your node too connect

root@node1:/home/ubuntu/.ssh# vim Docker-swarm-keypair.pem
root@node1:/home/ubuntu/.ssh# ls
Docker-swarm-keypair.pem  authorized_keys  known_hosts
root@node1:/home/ubuntu/.ssh# 

Be sure to connect to the private IP’s of these instances

Host node2
    HostName ec2-35-89-39-192.us-west-2.compute.amazonaws.com
    User ubuntu
    IdentityFile ~/root/.ssh//Docker-swarm-keypair.pem

Host node3
    HostName ec2-54-189-1-12.us-west-2.compute.amazonaws.com
    User ubuntu
    IdentityFile ~/root/.ssh/Docker-swarm-keypair.pem

:wq!

chmod 400 Docker-swarm-keypair.pem

https://www.tecmint.com/ssh-permission-denied-publickey/

root@node1:~/.ssh# ls authorized_keys
authorized_keys
root@node1:~/.ssh# ls -l authorized_keys
-rw------- 1 root root 566 Jul 18 20:35 authorized_keys
root@node1:~/.ssh# 

/root/.ssh/id_rsa.pub

I wuold follow this tutorial to create a public key
https://www.youtube.com/watch?v=sXNYIDi2qxE


 sudo chmod 600 ~/.ssh/id_rsa
$ sudo chmod 644 ~/.ssh/id_rsa.pub

Step 3. Joining the nodes for our swarm

From node1, open a new terminal window and we will use it to SSH into nodes 2 and 3.

Run sudo ssh node 2

sudo ssh node2

Repeat and run the same command for node 3 in a new terminal

Repeat for nodes 2 and 3 to make them workers.

sudo swarm init

Then add node1 to the swarm as a manager, run:

sudo docker swarm join — token (token) (manager-nodeIP)

sudo docker swarm join - token (token) (manager-nodeIP)
sudo swarm join --token 
sudo docker node ls

Step 4. Docker Compose file to create a Wordpress service

mkdir wordpress
cd wordpress
touch docker-compose.yaml

Step 5. Create 2 networks

ubuntu@node1:~wordpress$ sudo docker stack ls

docker network create -d overlay frontend
docker network create -d overlay backend

sudo docker service create --name mysql --network backend --replicas 1 --mount type=volume,source=mysql_data,destination=/var/lib/mysql -e MYSQL_ROOT_PASSWORD=larry -e MYSQL_DATABASE=wordpress mysql:latest

sudo docker service create --network frontend --name wordpress --replicas 3 -p 80:80 --mount type=volume,source=wordpress-data,target=/var/www/html wordpress:latest

The service will now be created in the last 2 commands

Step 6. Deploy docker-compose file using Docker Stack

Docker stack — is a management tool that groups and pools docker swarm resources together

sudo docker stack deploy -c docker-compose.yaml <name of stack>
sudo docker network ls
sudo docker service ls

Step 7. Connect to WordPress

<instance public IP:port>

Now let’s clean up our services after we have finished!

sudo docker service rm mysql wordpress

sudo docker network rm frontend backend

Please connect with me!

https://www.linkedin.com/in/sheenalynnnarciza/

https://github.com/Sheenanarciza