Continuing with the Kubernetes adventure I realized my setup had flaws. I couldn’t access the localhost within a multipass VM from my host (Mac), and I had difficulty setting up persistent storage for my cluster.

Here is a link to my first article Setting up your own Kubernetes Cluster with Multipass — A Step by Step Guide:
https://medium.com/gitconnected/setting-up-kubernetes-cluster-with-multipass-on-a-mac-a-step-by-step-guide-4645f8f30c45

Today we’ll continue with getting up and running the Persistent Volumes (PVs) and Persistent Volume Claims (PVCs), while bringing in Longhorn for some fancy, reliable storage magic. Get ready to supercharge our local Kubernetes setup!

To give Kubernetes apps disk storage, we are going to need Persistent Volumes. While HostPath and LocalPersistentVolumes work, they’re limited to single nodes and fail if the node does. For more reliable, redundant storage, Longhorn comes to the rescue!

Why Longhorn?

You have two main choices for self-hosted PersistentVolumes. Rook offers a powerful CEPH-based solution with block storage, but it needs extra drives, which isn’t great for smaller setups.

Longhorn, on the other hand, uses HostPath volumes and adds a layer to turn them into fault-tolerant PersistentVolumes. It spreads data across nodes to keep it safe from failures and manages storage that way, leaving room for system files and container caches.

Key Features of HostPath Volumes

HostPath volumes give containers direct access to files on the node, which is useful for sharing files or logs between container and the host.

Generally Not for Production: HostPath Volumes are not recommended for production because they tie your pod to a specific node. Due to immutable architecture of Kubernetes, the pod will not have access to the same HostPath volume if its rescheduled to a different node, which causes issues with data consistency.

BUT Longhorn while using HostPath volumes as part of its underlying architecture abstracts away many of the complexities and limitations of raw HostPath volumes to make them suitable for production environments.

Longhorn replicates data across multiple nodes, providing redundancy and fault tolerance. This actually makes it suitable for production environments where data availability is critical.

It automates many aspects of storage management, including provisioning, backups, and snapshots, simplifying storage operations.

Getting Longhorn Installed

Getting Longhorn up and running is a no brainer! It mostly takes care of itself, but you need to have open-iscsi and NFSv4 on each node. Instead of manually installing these on every single node, we are going to use DaemonSet to automate the process. DaemonSets are Kubernetes API objects that allow you to run Pods as a daemon on each of your Nodes. It’s quick and easy!

Just run this command within the kubemaster, all of our kubectl commands should run there:

kubectl apply -f https://raw.githubusercontent.com/longhorn/longhorn/v1.5.2/deploy/prerequisite/longhorn-iscsi-installation.yaml
kubectl apply -f https://raw.githubusercontent.com/longhorn/longhorn/v1.5.2/deploy/prerequisite/longhorn-nfs-installation.yaml

The installation might take a minute to wrap up. You can keep an eye on the progress with these commands:

kubectl get pod | grep longhorn-iscsi-installation

kubectl get pod | grep longhorn-nfs-installation

These commands show you the details of pods related to the Longhorn iSCSI installation by filtering out other pods from the list.

# You should see something like:

longhorn-nfs-installation-flnhs     1/1     Running   2 (10h ago)   14h
longhorn-nfs-installation-sfhmr     1/1     Running   2 (10h ago)   14h

After installing the dependencies, you’re ready to install Longhorn itself.

kubectl apply -f https://raw.githubusercontent.com/longhorn/longhorn/v1.5.2/deploy/longhorn.yaml

to monitor the progress use the — watch flag:

kubectl get pods -n longhorn-system --watch

Once ready, you should see a similar output:

NAME                                                READY   STATUS    RESTARTS      AGE
csi-attacher-6d8779d485-6nk6t                       1/1     Running   5 (9h ago)    14h
csi-attacher-6d8779d485-6qcng                       1/1     Running   2 (10h ago)   14h
csi-attacher-6d8779d485-dwxr9                       1/1     Running   1 (10h ago)   14h
csi-provisioner-69ffbb8c5b-2l7k7                    1/1     Running   2 (10h ago)   14h
csi-provisioner-69ffbb8c5b-5zcrr                    1/1     Running   3 (9h ago)    14h
csi-provisioner-69ffbb8c5b-ffbkg                    1/1     Running   4 (9h ago)    14h
csi-resizer-7b98f6fbc4-gs7zk                        1/1     Running   1 (10h ago)   14h
csi-resizer-7b98f6fbc4-hxshw                        1/1     Running   5 (9h ago)    14h
csi-resizer-7b98f6fbc4-ph6ms                        1/1     Running   2 (10h ago)   14h
csi-snapshotter-65d845d4f4-8bnp6                    1/1     Running   2 (10h ago)   14h
csi-snapshotter-65d845d4f4-gg5hl                    1/1     Running   5 (9h ago)    14h
csi-snapshotter-65d845d4f4-xg5mb                    1/1     Running   2 (10h ago)   14h
engine-image-ei-65224a4b-2fqjq                      1/1     Running   2 (10h ago)   14h
engine-image-ei-65224a4b-7brjg                      1/1     Running   1 (10h ago)   14h
instance-manager-84d479227caee7de34d45408517eeca0   1/1     Running   0             10h
instance-manager-ba2939744e154f4f95b7a2e527d76378   1/1     Running   0             10h
longhorn-csi-plugin-nd224                           3/3     Running   8 (10h ago)   14h
longhorn-csi-plugin-r48fx                           3/3     Running   4 (10h ago)   14h
longhorn-driver-deployer-74bf9cb677-f45fm           1/1     Running   3 (10h ago)   14h
longhorn-manager-kqk9k                              1/1     Running   1 (10h ago)   14h
longhorn-manager-x2ld9                              1/1     Running   2 (10h ago)   14h
longhorn-ui-fd949ccd4-24ghm                         1/1     Running   1 (10h ago)   14h
longhorn-ui-fd949ccd4-w2gdb                         1/1     Running   2 (10h ago)   14h

csi-attacher
The CSI Attacher is responsible for attaching volumes to nodes. It works in conjunction with the Kubernetes Controller Manager to ensure that volumes are properly attached to the correct nodes.

csi-provisioner
The CSI Provisioner handles the management of PersistentVolumes (PVs) based on PersistentVolumeClaims (PVCs). It ensures that storage resources are allocated and deallocated as required per the PVCs.

csi-resizer
The CSI Resizer manages the resizing of volumes. When a PVC requests a larger volume, the Resizer adjusts the size of the underlying PersistentVolume to accommodate the request.

csi-snapshotter
CSI Snapshotter handles the creation and management of volume snapshots. It enables backup and restore operations by taking snapshots of the volume data.

engine-image-ei
This component manages Longhorn engine images. Engine images are used to create and manage the Longhorn storage engines that interact with volumes.

instance-manager
Instance Managers are responsible for managing Longhorn volumes and their associated instances. They handle tasks like provisioning and maintaining the Longhorn volumes’ instances.

longhorn-csi-plugin
The Longhorn CSI Plugin interfaces with the Kubernetes CSI (Container Storage Interface) to provide Longhorn volumes to Kubernetes pods. It acts as a bridge between Kubernetes and the Longhorn storage system.

longhorn-driver-deployer
This pod deploys and manages the Longhorn driver components. It ensures that the necessary drivers are correctly set up and operational within the Kubernetes cluster.

longhorn-manager
The Longhorn Manager oversees the overall operation of the Longhorn system. It manages volume operations, health monitoring, and configuration tasks.

longhorn-ui
This pod runs the Longhorn user interface, providing a web-based dashboard for managing Longhorn storage, viewing volume status, and performing administrative tasks.

Check if Volumes are Working

To confirm that installation was successful and your storage is ready, create a PersistentVolumeClaim. This will prompt Longhorn to automatically provision a PersistentVolume, which takes care of that headache for you.

The same will work with deleting, Longhorn will remove the PersistentVolume once you delete the PersistentVolumeClaim.

At first I was manually setting up my own PVs and PVCs but various reasons Kubernetes had trouble managing them, especially deleting the volumes and I had to use all sorts of tricks to force Kubernetes do what I want.

For example a PV would get stuck Terminating and hang there, which meant I couldn’t create a PV with the same name and had to wait:

NAME          CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS        CLAIM                                         STORAGECLASS   VOLUMEATTRIBUTESCLASS   REASON   AGE
c-registry-pv 5Gi        RWO            Retain           Terminating   cybernetically-utils/container-registry-pvc   standard       <unset>

Longhorn makes all these problems go away, the modules I described above do all the management.

Here’s an example, this YAML defines a PersistentVolumeClaim named container-registry-pvc that requests 1GB of read-write storage from Longhorn, which will handle provisioning the actual PersistentVolume.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  namespace: container-registry-pvc
  name: longpvc
spec:
 accessModes:
   - ReadWriteOnce
 resources:
   requests:
     storage: 1Gi
 storageClassName: longhorn

Save this into “longhorn-pvc.yaml” and apply the Yaml file

kubectl apply -f longhorn-pvc.yaml

Inspect the claim by running:

kubectl describe pvc longpvc

You should see something like this, the most important part is the Status which should be Bound.

Name:          container-registry-pvc
Namespace:     default
StorageClass:  longhorn
Status:        Bound
Volume:        pvc-d462441d-0348-4325-a90c-f783a4481547
Labels:        <none>
Annotations:   pv.kubernetes.io/bind-completed: yes
               pv.kubernetes.io/bound-by-controller: yes
               volume.beta.kubernetes.io/storage-provisioner: driver.longhorn.io
               volume.kubernetes.io/storage-provisioner: driver.longhorn.io
Finalizers:    [kubernetes.io/pvc-protection]
Capacity:      1Gi
Access Modes:  RWO
VolumeMode:    Filesystem
Used By:       <none>
Events:        <none>

LongHorn Dashboard

Longhorn has a dashboard and you can access it by port forwarding from your multipass into your host, but there is trick with our setup.

I’ve spent many hours trying to figure this part out and apparently multipass has a problem with forwarding TCP6 requests that are coming from a host into the multipass VM.

I could ping my multipass VM, which I believe uses the TCP4, but I couldn’t access the localhost which uses TCP6, so I used the following technique to make it work.

First, port forward the long-horn frontend by running on the kubemaster.

kubectl port-forward service/longhorn-frontend -n longhorn-system 8080:80

Now the longhorn front end will be available for access within kubemaster, but we want to access it on a host through the browser. I have searched the internet for a couple of days and the only thing I found is that TCP6 requests are not forwarded by multipass. So here is the trick:

Install socat on kubemaster

sudo apt-get update
sudo apt-get install socat

And then have socat forward the TCP6 requests, thats it! At this point you are port forwarding from Kubernetes into the VM and from VM into the host.

kubemaster -- sudo socat "TCP6-LISTEN:8088",fork "TCP6:[::1]:8080"

For ease of use, you can run the above command via multipass exec from the host:

multipass exec -n kubemaster -- kubemaster -- sudo socat "TCP6-LISTEN:8088",fork "TCP6:[::1]:8080"

Once executed, now point the browser to 8088 port of your kubemaster, for example my browser URL is http://192.168.64.100:8088 and you should see the longhorn control panel:

Another way to expose the front end is by editing the service/longhorn-frontend:

kubectl edit service/longhorn-frontend -n longhorn-system

You should change the type to a NodePort as bellow:

Find the name of the kubernetes longhorn dashboard pod, run this command:

kubectl get pods -n longhorn-system

You should see a pod it starts with longhorn-ui for example:

longhorn-ui-fd949ccd4-24ghm                         1/1     Running   5 (93m ago)     3d13h
longhorn-ui-fd949ccd4-w2gdb                         1/1     Running   7 (93m ago)     3d13h

Whenever you modify the service type, you must delete the pod. Kuberntes creates a new one for you, essentially restarting the pod. So lets run these commands (make sure you use your pod uids you got from the previous command):

kubectl delete pod longhorn-ui-fd949ccd4-24ghm -n longhorn-system
kubectl delete pod longhorn-ui-fd949ccd4-w2gdb -n longhorn-system

Now to make sure Kubernetes restarted the pods run the command, you should see another set of pods starting:

kubectl get pods -n longhorn-system

Verify that the service is of a correct type and run this command:

kubectl get svc -n longhorn-system

You should see this:

NAME                          TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)        AGE
longhorn-admission-webhook    ClusterIP   10.101.1.14      <none>        9502/TCP       3d13h
longhorn-backend              ClusterIP   10.106.19.214    <none>        9500/TCP       3d13h
longhorn-conversion-webhook   ClusterIP   10.103.237.212   <none>        9501/TCP       3d13h
longhorn-engine-manager       ClusterIP   None             <none>        <none>         3d13h
longhorn-frontend             NodePort    10.96.49.169     <none>        80:32628/TCP   3d13h
longhorn-recovery-backend     ClusterIP   10.104.4.107     <none>        9503/TCP       3d13h
longhorn-replica-manager      ClusterIP   None             <none>        <none>         3d13h

Now open your favorite browser and navigate to a URL, where 192.168.64.100 is the kubemaster static IP.

https://192.168.64.100:32628

Wrap up

We’ve successfully set up Longhorn for Persistent Volumes in our Kubernetes cluster, boosting the local development setup with reliable, fault-tolerant storage.

With Longhorn’s automation and robust features, managing storage has never been easier!

If you’ve navigated any challenges or have questions, feel free to reach out.

Happy deploying!

Setting Up and Managing Persistent Volumes with Longhorn in Your Kubernetes Cluster was originally published in Level Up Coding on Medium, where people are continuing the conversation by highlighting and responding to this story.

Have you ever wanted to build your own Kubernetes setup? In this article I will take you on an ambitious journey to create a scalable infrastructure from scratch, a local AWS if you like. This endeavor was driven by a desire for hands-on experience and a deeper understanding of Kubernetes’ inner workings.

I will be using a Mac but I don’t think that matters much, because all of these commands and configuration can be useful on any platform where you can run Multipass because a lot of settings are applied to VMs themselves.

In Multipass, VMs are assigned dynamic IP addresses, that can change after a reboot. This behavior can disrupt a Kubernetes cluster, as changing IP addresses interfere with cluster communication and stability. Therefore, it is crucial to provision VMs with static IP addresses.

Setting up a new bridge with a static IP on the Mac

Here I go creating a new bridge with a static IP on my host.

To create a new bridge with a static IP on macOS, you can use the built-in ifconfig utility and modify network settings accordingly. Here is a step-by-step guide:

Create the Bridge Interface

Open Terminal.

Use the ifconfig command to create a new bridge interface. This example will use bridge0.

sudo ifconfig bridge0 create

Add the physical network interface to the bridge. Replace en0 with your network interface name.

sudo ifconfig bridge0 addm en0

Step 2: Configure the Bridge with a Static IP

Assign a static IP to the bridge. Replace 192.168.1.2 with your desired IP address and 255.255.255.0 with your subnet mask.

sudo ifconfig bridge0 inet 192.168.1.2 netmask 255.255.255.0 up

Set the default gateway if needed. Replace 192.168.1.1 with your gateway IP.

sudo route add default 192.168.1.1

Step 3: Persist Configuration

To make these changes persistent across reboots, you need to create a launch daemon.

Create a new launch daemon plist file in /Library/LaunchDaemons/. For example, com.local.bridge.plist.

sudo nano /Library/LaunchDaemons/com.local.bridge.plist

Add the following content to the plist file:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>com.local.bridge</string>
    <key>ProgramArguments</key>
    <array>
        <string>/usr/sbin/ifconfig</string>
        <string>bridge0</string>
        <string>create</string>
        <string>addm</string>
        <string>en0</string>
        <string>inet</string>
        <string>192.168.1.2</string>
        <string>netmask</string>
        <string>255.255.255.0</string>
        <string>up</string>
    </array>
    <key>RunAtLoad</key>
    <true/>
</dict>
</plist>

Set the appropriate permissions for the plist file.

sudo chown root:wheel /Library/LaunchDaemons/com.local.bridge.plist
sudo chmod 644 /Library/LaunchDaemons/com.local.bridge.plist

Load the plist file to apply the settings.

sudo launchctl load /Library/LaunchDaemons/com.local.bridge.plist

Step 4: Verify the Configuration

Check that the bridge is active and has the correct IP configuration.

ifconfig bridge0

This process sets up a network bridge with a static IP on macOS Monterey, and the configuration persists across reboots.

Setup a Control-Plane Instance

A control-plane is essentially the VM thats going to be the master within our cluster, it will coordinate and schedule jobs on other nodes within the cluster.

We need to launch a virtual machine instance using Multipass.

We are going to call it kubemaster and put it on the en0 network. We’ll allocate 10gb of disk space for the instance and 3gb RAM, 2CPU cores and thats going to be using the base name “Jammy” which reffers to the codename for Ubuntu 22.04 LTS (Jammy Jellyfish)

multipass launch --disk 10G --memory 3G --cpus 2 --name kubemaster --network name=en0,mode=manual,mac="02:42:ac:11:00:02" jammy

You should see the kubemaster spinning up and then
Launched: kubemaster

Run the command to see the kubemaster IP address:
multipass list or you could actually shell into the instance with multipass shell kubemaster

My IP address for kubemaster is 192.168.64.4. So the static IP should end with 100, but when setting up you can pick your own IP, so let’s configure the extra interface for the instance.

Just run this command, which executes multipass exec, which means it will execute this command within the kubemaster instance

multipass exec kubemaster -- sudo bash -c 'cat << EOF > /etc/netplan/10-custom.yaml
network:
  version: 2
  ethernets:
    extra0:
      dhcp4: no
      match:
        macaddress: "52:54:00:d5:d5:11"
      addresses: [192.168.64.100/24]
EOF'

Then run this to apply the configuration, which will run another command within the kubemaster instance

multipass exec -n kubemaster - sudo netplan apply

You should get this output, ignore any warnings

Go ahead and confirm that its working, run this command:

multipass info kubemaster

You should see this type of output:

Name: kubemaster
State: Running
Snapshots: 0
IPv4: 192.168.64.5
 192.168.64.100
Release: Ubuntu 22.04.4 LTS
Image hash: 769f0355acc3 (Ubuntu 22.04 LTS)
CPU(s): 2
Load: 0.00 0.00 0.00
Disk usage: 1.6GiB out of 9.6GiB
Memory usage: 180.8MiB out of 2.9GiB

Go ahead and ping these IPs to make sure everything is in order

ping 192.168.64.5
ping 192.168.64.100

You can also execute a command from within the instance and ping your local machine, make sure to replace the following IP with your local host IP

multipass exec -n kubemaster -- ping 192.168.132.63

PING 192.168.132.63 (192.168.132.63) 56(84) bytes of data.
64 bytes from 192.168.132.63: icmp_seq=1 ttl=64 time=0.835 ms
64 bytes from 192.168.132.63: icmp_seq=2 ttl=64 time=0.797 ms
64 bytes from 192.168.132.63: icmp_seq=3 ttl=64 time=0.735 ms
64 bytes from 192.168.132.63: icmp_seq=4 ttl=64 time=0.511 ms
64 bytes from 192.168.132.63: icmp_seq=5 ttl=64 time=0.627 ms
64 bytes from 192.168.132.63: icmp_seq=6 ttl=64 time=0.999 ms
64 bytes from 192.168.132.63: icmp_seq=7 ttl=64 time=0.826 ms
64 bytes from 192.168.132.63: icmp_seq=8 ttl=64 time=0.767 ms
64 bytes from 192.168.132.63: icmp_seq=9 ttl=64 time=0.758 ms
64 bytes from 192.168.132.63: icmp_seq=10 ttl=64 time=1.03 ms

Create both mules

Now we’ll go ahead and setup both of our mule nodes kubesmule101 & kubesmule102

Run these commands one after the other:

multipass launch --disk 10G --memory 3G --cpus 2 --name kubesmule01 --network name=en0,mode=manual,mac="44:33:00:4b:ba:e1" jammy

multipass launch --disk 10G --memory 3G --cpus 2 --name kubesmule02 --network name=en0,mode=manual,mac="44:33:00:4b:cd:ee" jammy

After that run these commands to make sure both instances have been created:

multipass info kubesmule01

multipass info kubesmule02

You should see this in the output

Notice the IPs I have here are 192.168.64.X, we’ll use this, now lets create an extra interface for both of these instances by running these commands. Here we are running the command within each instance that assigns it a static IP, I used 192.168.64.120 and 192.168.64.130:

multipass exec -n kubesmule01 -- sudo bash -c 'cat << EOF > /etc/netplan/10-custom.yaml
network:
  version: 2
  ethernets:
    extra0:
      dhcp4: no
      match:
        macaddress: "44:33:00:4b:ba:e1"
      addresses: [192.168.64.120/24]
EOF'


multipass exec -n kubesmule02 -- sudo bash -c 'cat << EOF > /etc/netplan/10-custom.yaml
network:
  version: 2
  ethernets:
    extra0:
      dhcp4: no
      match:
        macaddress: "44:33:00:4b:cd:ee"
      addresses: [192.168.64.130/24]
EOF'

Run these commands to apply the configuration:

multipass exec -n kubesmule01 -- sudo netplan apply

multipass exec -n kubesmule02 -- sudo netplan apply

Lets check if our instances have static IPs, run this:

multipass info kubesmule01
multipass info kubesmule02

You should see two IPs in the output of each command:

Name:           kubesmule01
State:          Running
Snapshots:      0
IPv4:           192.168.64.7
                192.168.64.120
Release:        Ubuntu 22.04.4 LTS
Image hash:     769f0355acc3 (Ubuntu 22.04 LTS)
CPU(s):         2
Load:           0.00 0.00 0.00
Disk usage:     1.6GiB out of 9.6GiB
Memory usage:   173.8MiB out of 2.9GiB

Name:           kubesmule02
State:          Running
Snapshots:      0
IPv4:           192.168.64.8
                192.168.64.130
Release:        Ubuntu 22.04.4 LTS
Image hash:     769f0355acc3 (Ubuntu 22.04 LTS)
CPU(s):         2
Load:           0.00 0.00 0.00
Disk usage:     1.6GiB out of 9.6GiB
Memory usage:   182.5MiB out of 2.9GiB

Now for the final test, lets ping all of these IPs from the local host and ping the local host from within each instance:

ping 192.168.64.7
ping 192.168.64.120

ping 192.168.64.8
ping 192.168.64.130

multipass exec -n kubesmule01 -- ping 192.168.132.63

multipass exec -n kubesmule02 -- ping 192.168.132.63

Configuring the local DNS

Now we have to change the hosts file inside each of the instances in order to configure the local DNS for each of the instances, you can do it one after the other or just open three Terminal tabs and do it in paralell:

multipass shell kubemaster
multipass shell kubesmule01
multipass shell kubesmule02

Run the following command in each instance:

sudo vi /etc/hosts

and add this information to the file. If you aren’t familiar with vi then press “a” once you enter the editor to start editing the file, once done press “Esc” and “:wq” to save and exit vi.

192.168.64.100 kubemaster
192.168.64.120 kubemule01
192.168.64.130 kubemule02

Now that we are done with setting up our instances, time to setup Kubernetes itself.

Finally, Setting up Kubernetes

Open up 3 terminals and shell inside each of the instances like this and run the following commands in each of the instances:

The following commands are used to configure a Linux system for Kubernetes (k8s) setup by ensuring necessary kernel modules are loaded and sysctl parameters are set correctly. Here’s a breakdown of what each part does:

Loading Kernel Modules (k8s.conf):

cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF

• This command writes the strings overlay and br_netfilter to the file /etc/modules-load.d/k8s.conf.
• These modules (overlay and br_netfilter) are required by Kubernetes for networking and container runtime management.

sudo modprobe overlay
sudo modprobe br_netfilter

• These commands manually load the kernel modules overlay and br_netfilter into the kernel immediately after adding them to the configuration file.
• modprobe is used to load kernel modules into the kernel dynamically.

Setting Sysctl Parameters (k8s.conf):

cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables  = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward                 = 1
EOF

• This command writes the specified sysctl parameters to the file /etc/sysctl.d/k8s.conf.
• These parameters (net.bridge.bridge-nf-call-iptables, net.bridge.bridge-nf-call-ip6tables, and net.ipv4.ip_forward) are required for Kubernetes networking and IP forwarding.

sudo sysctl --system

• This command applies the changes made in /etc/sysctl.d/k8s.conf immediately without needing a system reboot.
• It reads and applies sysctl settings from all files under /etc/sysctl.d/ and /run/sysctl.d/.

Verification:

• After configuring the modules and sysctl parameters, the script verifies if everything is set correctly.

lsmod | grep br_netfilter
lsmod | grep overlay

These commands check if the kernel modules br_netfilter and overlay are currently loaded.

sysctl net.bridge.bridge-nf-call-iptables net.bridge.bridge-nf-call-ip6tables net.ipv4.ip_forward

This command verifies that the sysctl parameters net.bridge.bridge-nf-call-iptables, net.bridge.bridge-nf-call-ip6tables, and net.ipv4.ip_forward are set to 1 as expected.

In summary, we’ve automated the configuration steps required to set up Linux for running Kubernetes, ensuring that necessary kernel modules are loaded and sysctl parameters are correctly configured. These steps are essential for Kubernetes to manage networking and IP forwarding effectively on the host system.

Now that the networking is setup, we can setup the runtime.

Setting up the Instance Runtime

We will be configuring containerd, an industry-standard container runtime recognized for its simplicity, robustness, and portability.

As a daemon, containerd oversees the entire container lifecycle on its host system, encompassing tasks from image transfer and storage to container execution and supervision, as well as low-level storage management and network attachments.

We’ll be using the latest version 1.7.14, so lets execute the following commands on every instance, but first you need to understand your current system architecture, execute “uname -m” to find out. Mine is x86_64, which means we’ll be installing the amd64 versions of containerd. You can find these links here https://github.com/containerd/containerd/releases

# Download containerd binaries:
curl -LO https://github.com/containerd/containerd/releases/download/v1.7.19/containerd-1.7.19-linux-amd64.tar.gz

# Extract containerd binaries to /usr/local/:
sudo tar Cxzvf /usr/local containerd-1.7.19-linux-amd64.tar.gz

# Download containerd systemd service file:
curl -LO https://raw.githubusercontent.com/containerd/containerd/main/containerd.service

# Move systemd service file to appropriate directory:
sudo mkdir -p /usr/local/lib/systemd/system/
sudo mv containerd.service /usr/local/lib/systemd/system/

# Configure containerd with default settings:
sudo mkdir -p /etc/containerd/
sudo containerd config default | sudo tee /etc/containerd/config.toml > /dev/null

# Modify containerd configuration file to enable systemd cgroup integration:
sudo sed -i 's/SystemdCgroup \= false/SystemdCgroup \= true/g' /etc/containerd/config.toml

# Reload systemd manager configuration:
sudo systemctl daemon-reload

# Enable and start containerd service:
sudo systemctl enable --now containerd

# Check status of containerd service:
systemctl status containerd

Here is the break down of these commands:

1. Download containerd binaries:

curl -LO https://github.com/containerd/containerd/releases/download/v1.7.19/containerd-1.7.19-linux-amd64.tar.gz

• We need to obtain the containerd binaries to install and configure containerd on these Linux instances. So we use curl to download the containerd binary

2. Extract containerd binaries to /usr/local/:

sudo tar Cxzvf /usr/local containerd-1.7.19-linux-amd64.tar.gz

• Here we use atar tool for handling archives to extract (x), compress (z), and show verbose output (v) of the contents of the containerd-1.7.14-linux-arm64.tar.gz file. The C flag ensures the operation is executed in /usr/local/
• By extracting the binaries to /usr/local/, you make them accessible system-wide, allowing containerd to be executed from anywhere on your system.

3. Download containerd systemd service file:

curl -LO https://raw.githubusercontent.com/containerd/containerd/main/containerd.service

• Again, we are using curl command to download the containerd.service file from the containerd GitHub repository (https://raw.githubusercontent.com/containerd/containerd/main/).
• The containerd.service file is needed to manage containerd as a systemd service, allowing it to start automatically and manage its lifecycle.

4. Move systemd service file to appropriate directory:

sudo mkdir -p /usr/local/lib/systemd/system/
sudo mv containerd.service /usr/local/lib/systemd/system/

• These commands create a directory (mkdir -p /usr/local/lib/systemd/system/) if it doesn't exist (-p flag), then move (mv) the containerd.service file to that directory.
• Placing containerd.service in /usr/local/lib/systemd/system/ ensures that systemd can locate and manage the containerd service effectively.

5. Configure containerd with default settings:

sudo mkdir -p /etc/containerd/
sudo containerd config default | sudo tee /etc/containerd/config.toml > /dev/null

• These commands create a directory (mkdir -p /etc/containerd/) if it doesn't exist, then generate the default configuration for containerd (containerd config default). The output (|) of this configuration command is redirected (tee) to /etc/containerd/config.toml. The tee command also outputs to /dev/null to discard unnecessary console output.
• This step ensures that containerd has a basic configuration file (config.toml) in place with default settings, which is necessary for its proper operation.

6. Modify containerd configuration file to enable systemd cgroup integration:

sudo sed -i 's/SystemdCgroup \= false/SystemdCgroup \= true/g' /etc/containerd/config.toml

• This command uses sed (stream editor) to replace (s) occurrences of SystemdCgroup = false with SystemdCgroup = true in the /etc/containerd/config.toml file.
• Enabling systemd cgroup integration ensures that containerd uses systemd for managing cgroups (control groups), which is the preferred method for managing resource limits and isolation in modern Linux distributions.

7. Reload systemd manager configuration:

sudo systemctl daemon-reload

• systemctl daemon-reload instructs systemd to reload its configuration files and units.
• After making changes to systemd service files or configurations, you need to reload systemd to apply those changes.

8. Enable and start containerd service:

sudo systemctl enable --now containerd

• systemctl enable --now containerd enables (enable) the containerd service to start on boot (--nowalso starts it immediately).
• Enabling the service ensures that containerd starts automatically when the system boots up (enable), and starting it immediately (--now) ensures that it is running without requiring a manual start.

9. Check status of containerd service:

systemctl status containerd

• systemctl status containerd displays the current status of the containerd service.
• This command is used to verify that containerd has started successfully and is running without any errors. Here is my output:

Now we need to install Runc

runc is a command-line tool for spawning and running containers according to the Open Container Initiative (OCI) specification. It is the industry-standard runtime used by container engines like Docker, containerd, and others to execute containers based on OCI-compliant container images.

You can find the latest release here: https://github.com/opencontainers/runc/releases/

Execute these commands on all 3 instances.

Here curl downloads run runtime into the home directory:

curl -LO https://github.com/opencontainers/runc/releases/download/v1.1.13/runc.amd64

This ensures that runc, a container runtime executable for the amd64 architecture, is properly installed and executable on the system:

sudo install -m 755 runc.amd64 /usr/local/sbin/runc

Install CNI plugins (https://www.cni.dev)

CNI (Container Network Interface) plugins are a set of specifications and plugins that define how network connectivity is established and managed for containers running on Linux-based systems.

CNI plugins provide a standardized way for container orchestrators (like Kubernetes, Docker, etc.) to configure networking for containers. They handle tasks such as assigning IP addresses, creating virtual network interfaces, setting up routing rules, and managing network policies.

You can find the latest versions here:
https://github.com/containernetworking/plugins/releases

Go ahead and run these commands on all 3 instances. These commands download and install the plugins within the instances.

curl -LO https://github.com/containernetworking/plugins/releases/download/v1.5.1/cni-plugins-linux-amd64-v1.5.1.tgz
sudo mkdir -p /opt/cni/bin
sudo tar Cxzvf /opt/cni/bin cni-plugins-linux-amd64-v1.5.1.tgz

Installing Kubernetes (kubeadm, kubelet and kubectl)

Kubernetes, also known as K8s, is an open source system for automating deployment, scaling, and management of containerized applications.

kubeadm

kubeadm is a command-line tool used for bootstrapping Kubernetes clusters. It facilitates the setup of a basic Kubernetes cluster by handling the initialization of the control plane, joining nodes to the cluster, and ensuring that the cluster meets the minimum requirements for Kubernetes operation.

System administrators and DevOps engineers use kubeadm during initial cluster setup, upgrades, and node management. It abstracts complex Kubernetes setup tasks into simpler commands.

kubelet

kubelet is an essential Kubernetes component running on each node in the cluster. It's responsible for managing the containers on the node, ensuring they run in the desired state specified by the Kubernetes manifests.

kubelet is critical for the operational health of each node in a Kubernetes cluster. It ensures that applications defined in Pods are correctly scheduled, launched, and maintained on the node.

kubectl

kubectl is the command-line tool used for interacting with the Kubernetes cluster. It allows users to deploy and manage applications, inspect and manage cluster resources, and view logs and metrics.

Developers, system administrators, and operators use kubectl extensively to interact with Kubernetes clusters. It provides a powerful interface for managing and troubleshooting applications and resources in Kubernetes.

Together, kubeadm, kubelet, and kubectl form the core toolset for deploying, managing, and operating Kubernetes clusters, catering to different aspects of cluster lifecycle management and application deployment in Kubernetes environments.

Run these commands on every instance:

# Update the local package database on your Linux system
sudo apt-get update

# Install necessary packages required for securely downloading packages over HTTPS
sudo apt-get install -y apt-transport-https ca-certificates curl gpg

# Download and install the Kubernetes package repository signing key
curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.29/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg

# Add the Kubernetes package repository to your system's package sources
echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.29/deb/ /' | sudo tee /etc/apt/sources.list.d/kubernetes.list

# Update the local package database again to include the newly added Kubernetes repository
sudo apt-get update

# Install Kubernetes components (kubelet, kubeadm, kubectl) on your system
sudo apt-get install -y kubelet kubeadm kubectl

# Prevent apt from automatically updating the specified packages
sudo apt-mark hold kubelet kubeadm kubectl

And now verify that the installation is a success by running these commands:

# Displays the version of kubeadm installed on your system
kubeadm version

# Displays the version of kubelet installed on your system
kubelet --version

# Displays the client version of kubectl installed on your system.
kubectl version --client

Now configure crictl to work with containerd

Configuring crictl to work with containerd involves specifying the runtime endpoint where containerd listens for container runtime requests. Let's break down why this configuration step is necessary and what it entails:

crictl is a command-line interface (CLI) tool for interacting with container runtimes that adhere to the Container Runtime Interface (CRI) specification.

It allows administrators and developers to manage containers and pods directly, inspect container runtime configuration, and perform troubleshooting tasks.

containerd communicates with tools like crictl over a Unix socket (unix:///var/run/containerd/containerd.sock).

This socket acts as a communication channel through which crictl sends commands to containerd and receives responses.

By configuring crictl with the runtime-endpoint option (sudo crictl config runtime-endpoint unix:///var/run/containerd/containerd.sock), you specify where crictl should send its commands related to container management.

This step ensures that crictl interacts with containerd correctly and can perform operations such as creating, inspecting, starting, stopping, and deleting containers managed by containerd.

Kubernetes uses crictl (via cri-containerd) as an interface to manage containers on nodes that run containerd.

Configuring crictl to point to containerd's Unix socket ensures that Kubernetes can orchestrate containers using containerd as the underlying runtime without issues.

This configuration step is essential for maintaining operational continuity and ensuring that container operations are managed correctly within Kubernetes clusters using containerd as the runtime engine.

Run this on every instance:

sudo crictl config runtime-endpoint unix:///var/run/containerd/containerd.socSe the controlplane node

Setting up the control-plane node

These commands should only be run on kubemaster. Kubemaster will control our mules and schedule to run the containers there.

The following command Initializes a new Kubernetes control plane. The apiserver-advertise-address parameter should be our Kubemaster static IP.

sudo kubeadm init --pod-network-cidr=10.244.0.0/16 --apiserver-advertise-address=192.168.64.100

When complete you should get the following output, which contains the kubeadm commands that you need to save to run on mule nodes.

When we run sudo kubeadm init, Kubernetes performs several crucial steps to set up the control plane:

• It generates the necessary TLS certificates needed for secure communication within the cluster.
• It starts essential components like the API Server, Controller Manager, and Scheduler.
• It generates a kubeconfig file for the kubectl command-line tool, which allows administrative access to the cluster.
• It generates a token that other nodes can use to join the cluster.

Post-Initialization Steps:

After running sudo kubeadm init, Kubernetes provides instructions and commands that you should run on worker nodes to join them to the cluster. These commands typically include using the kubeadm join command with a token and discovery token hash generated during initialization. Here is my output:

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

  export KUBECONFIG=/etc/kubernetes/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 192.168.64.100:6443 --token 6h6iro.dvvn76jr20pn9i61 \
 --discovery-token-ca-cert-hash sha256:17b8ad18d02f4c6bb5d6705273aafdf9460840dd88a53271192c001e96425d7e 

Configuring kubectl

kubectl is a command-line tool for interacting with Kubernetes clusters. This command is part of setting up kubectl to interact with the Kubernetes cluster that was initialized using sudo kubeadm init.

Once configured, kubectl commands can be executed directly by the user without needing to specify the configuration file location each time.

Run these commands on the kubemaster:

# Creates the .kube directory in the user's home directory
mkdir -p $HOME/.kube

# Copies the Kubernetes cluster configuration file (admin.conf) to the .kube/config file in the user's home directory.
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config

# Changes the ownership of the config file in .kube directory to the current user.
sudo chown $(id -u):$(id -g) $HOME/.kube/config

Now lets verify that cluster is reached via kubectl, execute on kubemaster:

kubectl -n kube-system get pods

Install Weave Net

Next we need to make sure our Pods can communicate with each other, which is why we must deploy a Container Network Interface (CNI) based Pod network add-on. We’ll use kubectl to initiate the service.

Execute the following command on kubemaster:

kubectl apply -f https://reweave.azurewebsites.net/k8s/v1.28/net.yaml

By now, the control plane node should be operational, with all pods in the kube-system namespace. Please verify this to ensure the control plane’s functionality and health. Run:

kubectl -n kube-system get pods

Time to join the mules into the cluster

Connect to each mule and execute the entire kubeadm join command that was copied earlier from the output of the kubeadm init command.

This step integrates each mule into the Kubernetes cluster's control plane, allowing them to join and contribute to cluster operations.

It guarantees that all nodes are synchronized and capable of participating in workload distribution and management tasks across the Kubernetes cluster.

This process is essential for achieving a functional and scalable Kubernetes environment where nodes collaborate seamlessly to run containerized applications.

sudo kubeadm join 192.168.64.100:6443 --token 6h6iro.dvvn76jr20pn9i61 \
 --discovery-token-ca-cert-hash sha256:17b8ad18d02f4c6bb5d6705273aafdf9460840dd88a53271192c001e96425d7e

Also, you can always get a new join command from control-pane instance by running this in the kubemaster, which will generate a new join token:

kubeadm token create --print-join-command

Now check that all the mules joined the cluster by running this command in the kubemaster:

kubectl get nodes

You should get this output:

Validating the Kubernetes setup

To ensure the Kubernetes setup functions correctly, verify by deploying an nginx pod within the cluster.

Execute the following commands on the Kubernetes master node:

kubectl run test-nginx --image=nginx

then run this command to check the status of the service:

kubectl get pod test-nginx

At first you will see the following output where the READY status is 0/1 but then after a few seconds you should get READY 1/1 just like I have it in the following output:

Congratulations! You have setup Kubernetes!!!

To stop the nginx pod that you deployed using kubectl run, you can delete the pod using kubectl delete pod. Here's how you can do it:

kubectl delete pod test-nginx

Kubernetes manages resources like pods, and deleting a pod stops the associated container(s) and removes the pod from the cluster.

And finally, the following diagram describes perfectly our current setup.

Enjoy it and Happy Kuberneting!

Setting up Kubernetes Cluster with Multipass on A Mac — A Step by Step Guide was originally published in Level Up Coding on Medium, where people are continuing the conversation by highlighting and responding to this story.

I’ve dedicated a few hours to enhancing the docker image build console output by adding color to various text fragments, and the result is fantastic. I am really excited to share it with you.

First things first, I whipped up a multi-stage Dockerfile for a fun little NodeJS project. I used NCC to bundle my whole NodeJS app and its modules into one slick .JS file, making my final Docker image way smaller. With NCC, I shrank it from a whopping 1GB to just 133.99MB!

Here is the file and a gist link, I’ve explained every bit of it in the code itself but here is a line by line explanation of it.
https://gist.github.com/jsmuster/70082c79d95d663a872d14a67166ff11

Stage 1: Builder

This is the part which creates a temporary Docker image used to compile our NodeJS application code into one single Javascript file, which is used in the next step of our multi-stage build.

Base Image

FROM node:lts-alpine as builder

• Uses the latest long-term support (LTS) version of Node.js with an Alpine Linux base image for the builder stage. Alpine is chosen for its small size and efficiency.

Environment Variable

ENV NODE_ENV=production

• Sets the NODE_ENV environment variable to production, indicating that the application is in production mode.

Working Directory

WORKDIR /usr/src/app

• Sets the working directory inside the container to /usr/src/app

Copy Package Files

COPY ["package.json", "package-lock.json*", "npm-shrinkwrap.json*", "./"]

• Copies package.json, package-lock.json, and npm-shrinkwrap.json (if they exist) to the working directory. These files are used to install the necessary dependencies.

Install Dependencies

RUN npm install --production

• Installs the production dependencies listed in the prior step.

Install NCC

RUN npm install -g @zeit/ncc

• Globally installs ncc (Node.js Compiler Collection), used to compile the Node.js project into a single file.

Copy Source Code

COPY . .

• Copies the rest of the application source code to the working directory.

Build Project

RUN export NODE_OPTIONS=--openssl-legacy-provider && ncc build ./bin/www -o dist

• Sets NODE_OPTIONS to use the OpenSSL legacy provider and runs ncc to compile the project located at ./bin/www into a single file output in the dist directory.

Remove Node Modules

RUN rm -rf node_modules

• Removes the node_modules directory to save space since it is no longer needed after the project is build.

Expose Port

EXPOSE 3000

• Informs Docker that the container listens on port 3000.

Stage 2: Build the actual Docker image

This is the part that builds the actual image

Base Image

FROM node:lts-alpine as backend

• Uses the same Node.js LTS Alpine image for our NodeJS application.

Environment Variable

ENV NODE_ENV=production

• Sets the NODE_ENV environment variable to production.

Working Directory

WORKDIR /usr/src/app

• Sets the working directory inside the container to /usr/src/app. You could change this to anything you like.

Copy Compiled Files

COPY --from=builder /usr/src/app/dist .

• Copies the compiled files from the dist directory of the builder docker image to the working directory of the current docker image.

Define Command

CMD ["node", "index.js"]

• Sets the default command to run the application using node index.js.

Now here is the actual script I wrote to enhance the Docker build output.

Because trying to read a detailed Docker build log without any color is like watching a mime perform in the dark — you know something is happening, but you have no idea what.

The process began with an attempt to color the initial dash and number pattern “#1”. Once this step was successful, I was able to proceed and convert the entire output accordingly. Here is a small snippet of the output:

I’ve shared the script on Github gists, so go ahead and use it as boiler plate and enhance it even further. Lets break this code down line by line:

https://gist.github.com/jsmuster/fcce009ddb35fa15f2f409b20c4c94f2

Shebang

#!/bin/sh

• This specifies that the script should be run using the sh shell. This is a part of every shell script by default, just make sure to have this at the top of the file, otherwise your scripts won’t execute.

ANSI Color Codes

# Regular text colors
BLACK='\033[0;30m'
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[0;33m'
BLUE='\033[0;34m'
MAGENTA='\033[0;35m'
CYAN='\033[0;36m'
WHITE='\033[0;37m'

# Bold text colors
BOLD_BLACK='\033[1;30m'
BOLD_RED='\033[1;31m'
BOLD_GREEN='\033[1;32m'
BOLD_YELLOW='\033[1;33m'
BOLD_BLUE='\033[1;34m'
BOLD_MAGENTA='\033[1;35m'
BOLD_CYAN='\033[1;36m'
BOLD_WHITE='\033[1;37m'

# Background colors
BG_BLACK='\033[40m'
BG_RED='\033[41m'
BG_GREEN='\033[42m'
BG_YELLOW='\033[43m'
BG_BLUE='\033[44m'
BG_MAGENTA='\033[45m'
BG_CYAN='\033[46m'
BG_WHITE='\033[47m'

NC='\033[0m' # No Color

• Defines various ANSI color codes for regular text, bold text, and background colors. We’re going to use these to color output in the console.

Colored Echo Functions

echo_red() {
    echo "${RED}$1${NC}"
}

echo_green() {
    echo "${GREEN}$1${NC}"
}

echo_yellow() {
    echo "${YELLOW}$1${NC}"
}

echo_blue() {
    echo "${BLUE}$1${NC}"
}

• Defines functions to print text in specific colors (red, green, yellow, and blue).

Colorize Output Function

colorize_output() {
    while IFS= read -r line; do
        # Check for the pattern "#[0-9]* CACHED" and colorize it
        if echo "$line" | grep -q "#[0-9]*"; then
            # Apply color to the matched pattern
            line=$(color_pattern "$line")
        fi
        echo "$line"
    done
}

• Reads lines from the input and checks for patterns to colorize using the color_pattern function. The IFS= read -r line loop reads lines without trimming whitespace.

Color Pattern Function

color_pattern() {

    local input="$1"
    local red='\\033[0;31m'

    local green='\\033[0;32m'
    local yellow='\\033[0;33m'
    local blue='\\033[0;34m'

    local black='\\033[0;30m'
    local magenta='\\033[0;35m'
    local cyan='\\033[0;36m'
    local white='\\033[0;37m'

    # Bold text colors
    local bold_black='\\033[1;30m'
    local bold_red='\\033[1;31m'
    local bold_green='\\033[1;32m'
    local bold_yellow='\\033[1;33m'
    local bold_blue='\\033[1;34m'
    local bold_magenta='\\033[1;35m'
    local bold_cyan='\\033[1;36m'
    local bold_white='\\033[1;37m'

    # Background colors
    local bg_black='\\033[40m'
    local bg_red='\\033[41m'
    local bg_green='\\033[42m'
    local bg_yellow='\\033[43m'
    local bg_blue='\\033[44m'
    local bg_magenta='\\033[45m'
    local bg_cyan='\\033[46m'
    local bg_white='\\033[47m'

    local reset='\\033[0m'
    local pattern="#[0-9]+"
    
    input=$(echo "$input" | sed -E "s/($pattern)/${bold_white}${bg_black}\1${reset}/g")

    pattern="DONE"
    
    input=$(echo "$input" | sed -E "s/($pattern)/${green}\1${reset}/g")

    pattern="exporting"
    
    input=$(echo "$input" | sed -E "s/($pattern)/${green}\1${reset}/g")

    pattern="writing image"
    
    input=$(echo "$input" | sed -E "s/($pattern)/${bold_black}\1${reset}/g")

    pattern="[0-9]+\.[0-9]+s"
    
    input=$(echo "$input" | sed -E "s/($pattern)/${blue}\1${reset}/g")

    pattern="CACHED"
    
    input=$(echo "$input" | sed -E "s/($pattern)/${yellow}\1${reset}/g")

    pattern="\[internal\]"
    
    input=$(echo "$input" | sed -E "s/($pattern)/${cyan}\1${reset}/g")

    pattern='\[builder [0-9]+/[0-9]+\]'

    input=$(echo "$input" | sed -E "s/(\[builder [0-9]+\/[0-9]+\])/${bold_green}\1${reset}/g")

    pattern='\[backend [0-9]+/[0-9]+\]'

    input=$(echo "$input" | sed -E "s/(\[backend [0-9]+\/[0-9]+\])/${magenta}\1${reset}/g")
    
    # Return the modified string
    echo "$input"
}

• This function holds local color variables because it needs to escape the strings, it applies various color codes to different patterns in the input string using sed. SED is a stream editor in Linux.

Initial Status Message

echo_green "Starting the build script..."

• Prints a message indicating the start of the build script in green.

Run Docker Build Function

run_docker_build() {
    docker image build --progress=plain -t cybernetically-web-backend:latest . 2>&1 | tee dockerbuild.log | colorize_output
}

• Defines a function to run the Docker build command and pipe its output to both a log file and the colorize_output function. This is the line that runs the docker build, you want to replace the “cybernetically-web-backend:latest” name of the image to something of your own. This will also output your build log into dockerbuild.log

Execute Docker Build

run_docker_build

• Calls the run_docker_build function to start the Docker build process with colorized output.

Final Status Message

echo_green "build script finished."

• Prints a message indicating the completion of the build script in green.

Weekend Docker Build Makeover: Tips and Tricks was originally published in Level Up Coding on Medium, where people are continuing the conversation by highlighting and responding to this story.

The previous article:

Re-create the RxJS Library from Scratch: Deep Dive into RxJS Observables

There are two types of operators in RxJS.

Creation Operators

These are standalone functions that create the Observable. For instance, we already implemented of, interval & range in our library. These methods create an Observable and return it for further use by Pipeable Operators.

Any process that generates or transforms data could be made into the Creation Operator.

Every Creation Operator follows the following model:

For example:

The “of” operator accepts a set of values and propagates them into the operator with a forEach loop

“interval” operator utilizes the setInterval method to propagate an incremented number at a set interval.

“range” operator in RxJS allows for the propagation of a sequence of values within a defined range

Pipeable Operators

These are operator functions that filter or transform a value. In practice they are chained together and for that reason they are configured through the pipe method, which is part of the Observable returned from a Creation Operator.

This is the interaction model used in the RxJS library, demonstrating how a value is propagated into a subscription.

Here is an example:
The first pipe filters the values by selecting only those that are odd (i.e., not divisible by 2). The second pipe modifies the filtered values by adding 10 to each. As a result, the sequence of the initial values being piped through the two pipes is [3, 5, 11, 13, 23].

of(3, 5, 8, 11, 13, 18, 23).pipe(
    filter(n => n % 2 === 1),
    map(n => n + 10)
  )
  .subscribe(result => console.log(result));

The console output is the following:

In order to propagate a value through a series of pipes within our custom rxjs library, we could use a reducer, which is what RxJS library uses, see my article on mastering the reducers here:

https://medium.com/@jsmuster/mastering-reducers-a-comprehensive-guide-for-everyone-298c23ad21c7

But we will employ a basic for loop and an iterator pattern to pass values through our pipes. In the iterator pattern, each transformation returns an object containing “done” and “value” properties. The done property signals when to stop the transformation from proceeding further.

for(const pipe of pipes)
{
  result = pipe(val);

  if(result.done == true)
  {
    break;
  }
  else
  {
    val = result.value;
  }

  count++;
};

Each operator must return a function that takes (val, index) as parameters and returns an object, following the Iterator model, with value and done properties. The done property indicates whether the value should continue to propagate within the operator chain.

Here is the ‘filter’ operator, which accepts a predicate function as its argument. This function is used to decide whether the value should continue to propagate through the pipable chain or not, based on the result of the predicate function.

In programming, a predicate is a function that takes one or more arguments and returns a boolean value, either true or false. Predicates are commonly used in functions like filtering or searching. They are called "predicates" because they essentially make a statement about the input that is either true or false, allowing the program to decide whether to include or exclude the input based on result of the predicate function.

rxjs.operators.filter = (predicate, thisArg) => {
  
  let ret;
  let efnfilter = (val, index) => {

    ret = predicate(val, index);

    if(ret === true)
    {
      return {value: val, done: false};
    }
    else
    {
      return {done: true};
    }
  };

  if(thisArg != null)
  {
    return efnfilter.bind(thisArg);
  }
  else
  {
    return efnfilter;
  }
};

This is the ‘map’ operator, which executes for each value produced and transforms the value.

Both filter and map accept ‘thisArg’ as a parameter which gets applied as “this” of that operand.

rxjs.operators.map = (transformer, thisArg) => {
  
  let ret;
  let efnmap = (val, index) => {
    
    ret = transformer(val, index);
    
    return {value: ret, done: false};
  };
  
  if(thisArg != null)
  {
    return efnmap.bind(thisArg);
  }
  else
  {
    return efnmap;
  }
};

You’ll find the take operator implementation in the code as well. At this point I implemented 3 pipes and we can successfully run the following example:

let rxjs = require('./Observable.js');
let { of } = rxjs;
let { filter, map, take } = rxjs.operators;

of(3, 5, 8, 11, 13, 18, 23).pipe(
    filter(n => n % 2 === 1),
    map(n => n + 10),
    take(3)
  )
  .subscribe(result => console.log(result));

Here is the RxJS library implementation:

And here is our custom rxjs library implementation:

We get the following output:

Conclusion

This article has provided a deep dive into the world of RxJS Operators by extending the custom RxJS library we created in the previous article.

We explored two types of operators, Creation Operators and Pipeable Operators, and their roles in processing data within an Observable. We also demonstrated how to implement these operators, using filter, map, and take as examples.

By understanding the core concepts and implementations of RxJS Operators, we can better appreciate the power and flexibility that RxJS brings to your JavaScript projects.

This knowledge will help you build more efficient and readable asynchronous code, ultimately enhancing the overall quality of your applications.

Check out my open source framework QQ:

https://github.com/jsmuster/qq

Check out my other articles

• Angular Routing in 5 Minutes
• Searching for a good JavaScript Developer — the Interview Questions
• Angular Injector In 5 Minutes

Level Up Coding

Thanks for being a part of our community! Before you go:

• 📰 View more content in the Level Up Coding publication
• 💰 Free coding interview course ⇒ View Course
• 🔔 Follow us: Twitter | LinkedIn | Newsletter

🚀👉 Join the Level Up talent collective and find an amazing job

Re-create the RxJS Library from Scratch (Part 2): Deep Dive into RxJS Operators was originally published in Level Up Coding on Medium, where people are continuing the conversation by highlighting and responding to this story.

Как же мы можем создать Observables с нуля? Давайте посмотрим на документацию RxJS для рецепта, как вы видите, класс Observable расширяет Subscribable, а Subscribable — это интерфейс с методом subscribe, поэтому наш Observable должен иметь метод subscribe, верно?

Конструктор — еще один важный аспект, в нем есть необязательный параметр, который представляет собой функцию, вызываемую при подписке на Observable.

Эта функция получает объект-подписчик типа Observer, который может использоваться для передачи новых значений с помощью метода next, сигнализирования об ошибке с помощью метода error, или указания успешного завершения с помощью метода complete.

На основании вышеуказанной информации Observable должен работать следующим образом:

let obs = rxjs.Observable.create((subscriber) => {
  
  subscriber.next(1);
  subscriber.next(2);
  subscriber.next(3);

subscriber.complete();
});

obs.subscribe((value) => {
  console.log(value);
});

1. Создайте Observable и передайте метод в конструктор, который будет отправлять значения в подписчик Observable.
2. Наконец, подпишитесь на Observable, передав метод, который будет получать значения.

Метод subscribe вызывает функцию-конструктор, переданную в Observable. Это означает, что подписка на Observable — это момент, когда он начинает выполнять свою логику, а не во время создания, как обычно полагают. Метод конструктора сохраняется как ссылка и ожидает, когда будет вызван метод subscribe.

В моем примере я передаю функцию в конструктор Observable, которая выполняется немедленно при подписке, НО обычно она предоставляется реализацией библиотеки, которая указывает, что будет излучать Observable и когда оно будет это делать. И это часто асинхронно.

Так, например, в реальном сценарии этот сервис выполнит метод subscribe асинхронно только тогда, когда данные успешно получены из HTTP.

getConfigFromServer() {

this.configService.getConfig()
    .subscribe((data: Config) => this.config = {
        heroesUrl: data.heroesUrl,
        textfile:  data.textfile,
        date: data.date,
    });

}

Или вот Observable, который будет отправлять значение каждую секунду после подписки на него:

var observable = rxjs.Observable.create((subscriber) =>{
  let count = 0;
  let interval;
         
   interval = setInterval(()=>{
        subscriber.next('Hello world (' + count++ + ')');
        
        if(count > 5)
        {
         clearInterval(interval);
          
          subscriber.complete();
    }
   }, 1000);

});

Теперь давайте реализуем Observable и, наконец, выполним эти примеры кода.

Вот пример кода, использующий библиотеку rxjs для выполнения его:
https://jsfiddle.net/jsmuster/tgkc05vy/

А вот моя версия с собственной реализацией Observable:
https://jsfiddle.net/jsmuster/u39t6bh8/

Обе функции работают одинаково. Мы получаем следующий вывод в консоль в обоих примерах:

Теперь давайте рассмотрим пример, где Observable отправляет значение каждую секунду:

Мы получаем следующий вывод в консоль:

С помощью всего 80 строк кода нам удалось достичь базовой функциональности Observable, и все работает отлично. Фантастически!

Давайте повеселимся и улучшим функциональность! Мы можем создать упрощенную версию библиотеки Observables, которая удовлетворяет большинству наших потребностей, не включая полный пакет RxJS в наши проекты.

Следующий отрывок является частью документации RxJS. Чтобы это работало, нам нужно добавить метод “of”, как показано в следующем примере из документации RxJS.

Я реализовал оператор “of” в обновленной версии и внес некоторые изменения в то, как методы выполняются в Observable. По сути, я вызываю их через обертывающий метод через “call”, чтобы убедиться, что объект подписчика является “this” в методах next, complete и error. Посмотрите на следующий пример:

sub.complete = () => {
  subscriber.complete.call(subscriber);
};

Вы должны получить следующий вывод в консоли при его запуске:

На странице RxJS Observable есть еще один пример поведения, который использует метод “interval”. Этот метод возвращает объект подписки, который отписывается через 2,5 секунды с помощью функции setTimeout.

Для этого необходимо включить оператор interval, который генерирует подписку, от которой мы позже можем отписаться.

Вот пример работы с использованием стандартной библиотеки RxJS:

Здесь мы имеем экземпляр нашей собственной библиотеки rxjs в действии, как показано в этом примере:

Удивительно, что оба примера будут производить вывод — это демонстрирует, насколько расширилась наша собственная библиотека rxjs. Теперь мы можем использовать оператор interval для отправки чисел через Observable и отписываться при необходимости.

В заключение, создание собственной библиотеки Observable с нуля — отличное упражнение, чтобы понять внутреннее устройство Observables и то, как они обрабатывают асинхронные потоки данных. С помощью всего нескольких строк кода мы смогли достичь базовой функциональности Observable и улучшить ее, чтобы она удовлетворяла большинству потребностей.

Красота Observables заключается в их гибкости и простоте использования. Они могут использоваться для обработки потоков данных в различных сценариях, от простых эмиссий данных до сложных сетевых запросов. Понимая, как работают Observables, мы можем открыть для себя целый новый уровень функциональности в наших приложениях.

Спасибо за чтение, и я надеюсь, что этот статья дала вам лучшее понимание Observables и того, как создать собственную библиотеку Observable.

Проверьте мой фреймворк с открытым исходным кодом QQ:

https://github.com/jsmuster/qq

Проверьте мои другие статьи.

• Angular Routing in 5 Minutes
• Angular Injector In 5 Minutes
• Searching for a good JavaScript Developer — the Interview Questions

Пожалуйста, поставьте этой статье 👏 и поделитесь ею!

Então, como fazemos Observables do zero? Vamos dar uma olhada na documentação do RxJS para uma receita, como você pode ver, uma classe Observable estende um Subscribable, e Subscribable é uma interface com o método subscribe, então nosso Observable deve ter um método subscribe, estou certo?

O construtor é outro aspecto importante, ele tem um parâmetro opcional, que é uma função que é chamada quando o Observable é inscrito.

Essa função recebe um objeto de assinante do tipo Observer, que pode ser usado para emitir novos valores usando o método next, ou sinalizar um erro usando o método error, ou indicar a conclusão bem-sucedida usando o método complete.

Com base nas informações acima, o Observable deve funcionar assim:

let obs = rxjs.Observable.create((subscriber) => {
  
  subscriber.next(1);
  subscriber.next(2);
  subscriber.next(3);

subscriber.complete();
});

obs.subscribe((value) => {
  console.log(value);
});

1. Crie o Observable e passe um método para o construtor que irá despachar os valores para o assinante do Observable.
2. Finalmente, inscreva-se no Observable passando um método que receberá os valores.

O método subscribe invoca a função construtora passada para o Observable. Isso significa que a inscrição em um Observable é o ponto em que ele começa a executar sua lógica, em vez de durante a criação, como é comumente acreditado. O método construtor é salvo como uma referência e aguarda o método de inscrição para executá-lo.

No meu exemplo, estou passando uma função para o construtor do Observable, que é executada imediatamente quando nos inscrevemos, MAS geralmente é fornecida por uma implementação de biblioteca que especifica o que o Observable emitirá e quando o emitirá. E muitas vezes é assíncrono.

Por exemplo, no cenário real, o seguinte serviço executará o método de inscrição de forma assíncrona, somente quando os dados forem recebidos de um serviço HTTP com sucesso.

getConfigFromServer() {

this.configService.getConfig()
    .subscribe((data: Config) => this.config = {
        heroesUrl: data.heroesUrl,
        textfile:  data.textfile,
        date: data.date,
    });

}

ou aqui está um Observable que emitirá um valor a cada segundo depois que nos inscrevermos nele:

var observable = rxjs.Observable.create((subscriber) =>{
  let count = 0;
  let interval;
         
   interval = setInterval(()=>{
        subscriber.next('Hello world (' + count++ + ')');
        
        if(count > 5)
        {
         clearInterval(interval);
          
          subscriber.complete();
    }
   }, 1000);

});

Agora, vamos implementar o Observable e finalmente executar esses exemplos de código.

Aqui está o código de amostra usando a biblioteca rxjs para executá-lo:
https://jsfiddle.net/jsmuster/tgkc05vy/

E aqui está minha versão com implementação personalizada do Observable:
https://jsfiddle.net/jsmuster/u39t6bh8/

Ambos funcionam da mesma maneira. Obtemos a seguinte saída de console em ambos os exemplos:

Agora, vamos dar uma olhada no exemplo em que o Observable está emitindo um valor a cada segundo:

Obtemos a seguinte saída de console:

— 19
Com apenas 80 linhas de código, conseguimos obter funcionalidades básicas de Observable e tudo parece estar funcionando perfeitamente. Fantástico!

Vamos nos divertir e melhorar a funcionalidade! Podemos criar uma versão enxuta da biblioteca de Observables que atenda à maioria dos nossos casos de uso, sem precisar incluir o pacote RxJS completo em nossos projetos.

O trecho a seguir faz parte da documentação do RxJS. Para fazer isso funcionar, precisamos adicionar o método “of”, conforme demonstrado no seguinte exemplo da documentação do RxJS.

Implementei o operador “of” na versão atualizada e fiz algumas modificações na forma como os métodos são executados dentro do Observable. Basicamente, eu os chamo por meio de um método de envoltório via uma “chamada” para garantir que o objeto assinante seja “this” nos métodos next, complete e error. Confira o seguinte exemplo:

sub.complete = () => {
  subscriber.complete.call(subscriber);
};

Você deve obter o seguinte log de console ao executá-lo:

Na página Observable do RxJS, há outra ilustração de comportamento que usa o método “interval”. Esse método retorna um objeto de assinatura que é cancelado após 2,5 segundos usando a função setTimeout.

Para isso funcionar, precisamos incluir o operador interval que gera a assinatura, da qual podemos cancelar a inscrição posteriormente.

Aqui está o exemplo funcionando usando a biblioteca RxJS padrão:

Aqui temos uma instância de nossa biblioteca personalizada do rxjs em ação, conforme demonstrado por este exemplo:

É incrível que ambos produzirão saída — isso demonstra o quanto nossa biblioteca personalizada do rxjs se expandiu. Agora podemos usar o operador interval para enviar números através do Observable e cancelar a inscrição quando necessário.

Em conclusão, criar uma biblioteca de Observable personalizada do zero é um ótimo exercício para entender o funcionamento interno dos Observables e como eles lidam com fluxos de dados assíncronos. Com apenas algumas linhas de código, conseguimos obter funcionalidades básicas de Observable e melhorá-las para atender à maioria dos casos de uso.

A beleza dos Observables está em sua flexibilidade e facilidade de uso. Eles podem ser usados para lidar com fluxos de dados em uma variedade de cenários, desde emissões simples de dados até solicitações de rede complexas. Ao entender como os Observables funcionam, podemos desbloquear um novo nível de funcionalidade em nossas aplicações.

Obrigado por ler e espero que este artigo tenha lhe dado uma melhor compreensão dos Observables e como criar sua própria biblioteca personalizada de Observable.

Confira meu framework de código aberto QQ:

https://github.com/jsmuster/qq

Confira meus outros artigos

• Angular Routing in 5 Minutes
• Angular Injector In 5 Minutes
• Searching for a good JavaScript Developer — the Interview Questions

Por favor, dê um 👏 no artigo e compartilhe-o!

في هذه المقالة، سأستكشف مفهوم Observables من خلال إنشاء مكتبة صغيرة من البداية. سنغوص من خلال هذه التمرينات في العمليات الداخلية للobservables وكيفية إنشائها، وكيفية استخدامها لمعالجة تدفقات البيانات الغير متزامنة.

فكيف يمكننا إنشاء Observables من البداية؟ دعونا نلقي نظرة على توثيق RxJS للحصول على وصف، حيث يوضح الصف Observable أنه يمتد من Subscribable، و Subscribable هو واجهة تحتوي على طريقة subscribe، لذلك يجب أن يحتوي Observable لدينا على طريقة subscribe، أليس كذلك؟

المُنشئ هو جانب آخر مهم، حيث يحتوي على معامل اختياري، وهو دالة يتم استدعاؤها عندما يتم الاشتراك في Observable.

تتلقى هذه الدالة كائن المشترك من نوع Observer، الذي يمكن استخدامه لإرسال قيم جديدة باستخدام الطريقة next، أو إشارة الخطأ باستخدام الطريقة error، أو إشارة الانتهاء بنجاح باستخدام الطريقة complete.

استنادًا إلى المعلومات السابقة، يجب أن يعمل Observable على النحو التالي:

let obs = rxjs.Observable.create((subscriber) => {
  
  subscriber.next(1);
  subscriber.next(2);
  subscriber.next(3);

subscriber.complete();
});

obs.subscribe((value) => {
  console.log(value);
});

1. إنشاء Observable وتمرير الدالة إلى المنشئ التي ستقوم بإرسال القيم إلى المشترك في الObservable.
2. أخيرًا، الاشتراك في ال Observable من خلال تمرير الدالة التي ستستقبل القيم.

طريقة المُنشئ تحفظ كمرجع، وتنتظر طريقة الاشتراك لتنفيذها.

في مثالي هذا، أنا أقوم بتمرير دالة إلى منشئ Observable، والتي تُنفذ مباشرة عند الاشتراك، ولكن عادةً ما توفرها المكتبة التنفيذية التي تحدد ما سيتم إرساله من Observable ومتى سيتم إرساله، وغالبًا ما تكون غير متزامنة.

على سبيل المثال، في السيناريو الحقيقي، ستقوم الخدمة التالية بتنفيذ طريقة الاشتراك بشكل غير متزامن، فقط عندما يتم استقبال البيانات بنجاح من خدمة HTTP.

getConfigFromServer() {

this.configService.getConfig()
    .subscribe((data: Config) => this.config = {
        heroesUrl: data.heroesUrl,
        textfile:  data.textfile,
        date: data.date,
    });

}

أو هناك Observable سيقوم بإرسال قيمة كل ثانية بعد الاشتراك فيه:

var observable = rxjs.Observable.create((subscriber) =>{
  let count = 0;
  let interval;
         
   interval = setInterval(()=>{
        subscriber.next('Hello world (' + count++ + ')');
        
        if(count > 5)
        {
         clearInterval(interval);
          
          subscriber.complete();
    }
   }, 1000);

});

الآن، دعونا نقوم بتنفيذ Observable وننفذ هذه الأمثلة.

إليك الكود المستخدم في مكتبة rxjs لتنفيذ ذلك:
https://jsfiddle.net/jsmuster/tgkc05vy/

وإليك نسختي من مكتبة Observable المُنفذة بشكل مخصص:
https://jsfiddle.net/jsmuster/u39t6bh8/

كلاهما يعمل بنفس الطريقة. يتم الحصول على النتائج التالية في كل من الأمثلة:

دعونا الآن نلقي نظرة على المثال الذي يُرسل فيه Observable قيمة كل ثانية:

نحصل على النتيجة التالية على مخرجات الكونسول:

بمجرد استخدام 80 سطرًا من الكود، تمكنا من تحقيق وظائف ال Observable الأساسية وكل شيء يبدو يعمل بشكل ممتاز. رائع!

يعرض المقتطف التالي من توثيق RxJS للعمل، ولجعل هذا يعمل، يجب علينا إضافة طريقة “of” كما هو موضح في المثال التالي من توثيق RxJS.

لقد نفذت عامل “of” في النسخة المحدثة، وقمت بعمل بعض التعديلات على كيفية تنفيذ الطرق داخل Observable. أنا ببساطة أقوم بإستدعائهم عبر طريقة الإتصال “call” للتأكد من أن كائن المشترك هو “this” داخل طرق next، complete و error. تحقق من المثال التالي:

لجعل هذا يعمل، يجب علينا إضافة طريقة “of” كما هو موضح في المثال التالي من توثيق RxJS. لقد قمت بتنفيذ عامل “of” في الإصدار المحدث، وقمت ببعض التعديلات على كيفية تنفيذ الطرق داخل Observable. بالأساس، أقوم بتنفيذها من خلال طريقة قابلة للتخصيص عن طريق “call” للتأكد من أن كائن المشترك هو “this” داخل طرق next و complete و error. تحقق من المثال التالي:

sub.complete = () => {
  subscriber.complete.call(subscriber);
};

يجب عليك الحصول على النتيجة التالية عند تشغيله:

على صفحة RxJS Observable، هناك رسم توضيحي آخر للسلوك الذي يستخدم عامل “interval”. هذا العامل يعيد كائن اشتراك يتم إلغاء اشتراكه بعد 2.5 ثانية باستخدام دالة setTimeout.

لجعل هذا يعمل، يجب علينا تضمين عامل الفاصل الزمني “interval” الذي يولد الاشتراك، والذي يمكننا فيما بعد إلغاء اشتراكنا منه.

فيما يلي المثال الذي يعمل باستخدام مكتبة rxjs القياسية:

هنا لدينا مثالًا على تطبيق مكتبة rxjs المخصصة لدينا، كما هو موضح في هذا المثال:

مدهش كيف يمكن لكليهما أن ينتجا الإخراج — وهذا يوضح مدى توسع مكتبة rxjs المخصصة لدينا. يمكننا الآن استخدام عامل الفاصل الزمني لإرسال الأرقام من خلال Observable والإلغاء الاشتراك عند الضرورة.

في الخلاصة، إن إنشاء مكتبة Observable مخصصة من الصفر هو تمرين رائع لفهم العمليات الداخلية لل Observables وكيفية معالج

جمال Observables يكمن في مرونتها وسهولة استخدامها. يمكن استخدامها لمعالجة تدفقات البيانات في مجموعة متنوعة من السيناريوهات، بدءًا من الإرسالات البسيطة للبيانات إلى الطلبات الشبكية المعقدة. من خلال فهم كيفية عمل Observables، يمكننا فتح مستوى جديد كليًا من الوظائف في تطبيقاتنا.

شكرًا لك على القراءة، وآمل أن يكون هذا المقال قد أعطاك فهمًا أفضل لل Observables وكيفية إنشاء مكتبة Observable المخصصة الخاصة بك.

تحقق من إطار العمل المفتوح المصدر الخاص بي QQ:

https://github.com/jsmuster/qq

تحقق من مقالاتي الأخرى.

• Angular Routing in 5 Minutes
• Angular Injector In 5 Minutes
• Searching for a good JavaScript Developer — the Interview Questions

يرجى إعطاء المقال بعض الإعجاب ومشاركته!

এই নিবন্ধে, আমি শুরু থেকে একটি ছোট্ট লাইব্রেরি তৈরি করে পর্যবেক্ষণ করবো অবজারভেবলগুলোর ধারণার উপর। এই অনুশীলনের মাধ্যমে, আমরা অবজারভেবলগুলোর আন্তরিক কার্যকিণী পর্যবেক্ষণ করব, যেখানে তারা কিভাবে তৈরি করা যায় এবং অসিনক্রন ডেটা স্ট্রিমগুলি হ্যান্ডেল করার জন্য কিভাবে ব্যবহার করা যায়।

এখন আমরা কীভাবে শুরু থেকে ওবসারভেবল তৈরি করতে হবে তা নিয়ে চিন্তা করব? রিসিপি পেতে আমরা এরকম কিছু দেখতে পারি যেমন RxJS ডকুমেন্টেশনে, যেখানে ওবসারভেবল ক্লাস একটি সাবস্ক্রাইবেবল ক্লাসের একটি এক্সটেনশন। এটি একটি ইন্টারফেস সাবস্ক্রাইব মেথড সহ যা একটি সাবস্ক্রাইব মেথড থাকা উচিত। তাই আমাদের ওবসারভেবলের সাবস্ক্রাইব মেথড থাকা উচিত, আমি সঠিক বলছি না?

কনসট্রাক্টর হল একটি গুরুত্বপূর্ণ দিক, যা একটি ঐচ্ছিক প্যারামিটার দিয়ে থাকে, যা সাবস্ক্রাইব করা হলে ওবসারভেবলে কল করা হয়।

এই ফাংশনটি একটি অবজারভার টাইপের সাবস্ক্রাইবার অবজেক্ট পেয়ে এসে নতুন মান এমিট করতে এটি ব্যবহার করা যায় এক্সট মেথডটি ব্যবহার করে, অথবা ত্রুটি সংকেত ব্যবহার করে ত্রুটি সংকেত করতে ব্যবহার করা যায়, অথবা সফলভাবে সমাপ্তি নির্দেশ করতে ব্যবহার করা যায়।

উপরের তথ্যের উপর ভিত্তি করে অবজারভেবল এই ভাবে কাজ করবে:

let obs = rxjs.Observable.create((subscriber) => {
  
  subscriber.next(1);
  subscriber.next(2);
  subscriber.next(3);

subscriber.complete();
});

obs.subscribe((value) => {
  console.log(value);
});

1. অবজারভেবলটি তৈরি করুন এবং অবজেক্টটির সাবস্ক্রাইবারে মান প্রেরণ করার জন্য কনস্ট্রাক্টরে একটি মেথড পাস করুন।
2. আবহাওয়ানি ডেটা স্ট্রিমগুলি হ্যান্ডেল করতে ব্যবহৃত হয়ে ওবসার্ভেবলে সাবস্ক্রাইব করুন পাসিং একটি মেথড।

Observable তৈরি করার জন্য আমরা কীভাবে কাজ করব? একটি রেসিপির জন্য RxJS ডকুমেন্টেশন দেখি, যেখানে দেখা যায় যে Observable ক্লাস একটি Subscribable এর একটি এক্সটেন্ড করে এবং Subscribable হল একটি ইন্টারফেস যেখানে subscribe মেথড আছে, সুতরাং আমাদের Observable একটি subscribe মেথড থাকতে হবে। ঠিক আছি?

কনসট্রাক্টর আরেকটি গুরুত্বপূর্ণ দিক, এতে একটি ঐচ্ছিক প্যারামিটার আছে, যা হল যখন Observable সাবস্ক্রাইব করা হয় তখন কল করা হয়।

এই ফাংশনটি একটি Observer টাইপের subscriber অবজেক্ট পেয়ে, next মেথডটি ব্যবহার করে নতুন মান এমিট করতে পারে বা এরর মেথডটি ব্যবহার করে ত্রুটি সংকেত করতে পারে বা সফল সমাপ্তির জন্য সম্পূর্ণ মেথডটি ব্যবহার করতে পারে।

getConfigFromServer() {

this.configService.getConfig()
    .subscribe((data: Config) => this.config = {
        heroesUrl: data.heroesUrl,
        textfile:  data.textfile,
        date: data.date,
    });

}

উপরের তথ্যের উপর ভিত্তি করে Observable এমনভাবে কাজ করবে:

var observable = rxjs.Observable.create((subscriber) =>{
  let count = 0;
  let interval;
         
   interval = setInterval(()=>{
        subscriber.next('Hello world (' + count++ + ')');
        
        if(count > 5)
        {
         clearInterval(interval);
          
          subscriber.complete();
    }
   }, 1000);

});

এখন চলুন আমরা Observable টি ইমপ্লিমেন্ট করে এবং শেষমেষ কোড স্যাম্পলগুলি এক্সিকিউট করি।

রিএক্সজেএস লাইব্রেরি ব্যবহার করে এরকম স্যাম্পল কোডঃ
https://jsfiddle.net/jsmuster/tgkc05vy/

এবং এখানে আমার কাস্টম Observable ইমপ্লিমেন্টেশনসহ একই ফাংশনালিটি প্রদর্শিত হলঃ
https://jsfiddle.net/jsmuster/u39t6bh8/

উভয়ই একই ফাংশন করে। উভয় স্যাম্পলে আমরা নিম্নলিখিত কনসোল লগ পেয়েছি:

এখন আসুন আমরা যে উদাহরণটি নিয়ে কাজ করি, যেটি Observable কোনও সময় একটি মান এমিট করবে।

একটি মান এমিট হওয়া সময় আমরা নিম্নলিখিত কনসোল আউটপুট পেয়েছি:

মাত্র 80 লাইন কোড দিয়ে, আমরা বেসিক অবজারভেবল ফাংশনালিটি প্রাপ্ত করেছি এবং সবকিছু একটুখানি ভালভাবে কাজ করছে। অসাধারণ!

চলুন কিছু মজা করি এবং ফাংশনালিটি উন্নয়ন করি! আমরা আমাদের প্রকল্পগুলিতে সম্ভাব্যতম বেশি ব্যবহৃত কেসের জন্য সম্পূর্ণ আরএক্সজেএস প্যাকেজটি যোগ করতে না হয়ে, আমরা অবজারভেবল লাইব্রেরির একটি হালকা সংস্করণ তৈরি করতে পারি।

নিম্নলিখিত স্নিপেটটি আরএক্সজেএস নথিতে অংশ হয়, এটি কাজ করার জন্য, আমাদের কল করতে হবে “of” মেথডটি এর উদাহরণটি নির্দেশ করে।

আমি উন্নয়নকৃত সংস্করণে “of” অপারেটরটি বাস্তবায়ন করেছি এবং অবজারভেবলের ভিতরে মেথডগুলি কীভাবে নিষ্পাদিত হয় তা উন্নয়ন করেছি। মূলতঃ, আমি একটি “কল” মাধ্যমে এগুলি একটি ওয়্রাপার মাধ্যমে ডাক করি যাতে “পরবর্তী”, “সম্পূর্ণ” এবং “ত্রুটি” মেথডগুলির মধ্যে সাবস্ক্রাইবার অবজেক্ট “এই” হয়। নিচের উদাহরণটি দেখুন:

sub.complete = () => {
  subscriber.complete.call(subscriber);
};

আপনি এটি চালানোর সময় নিম্নলিখিত কনসোল লগ পাবেন:

RxJS অবজারভেবল পৃষ্ঠাতে, “ইন্টারভেল” মেথড ব্যবহার করে আরও একটি আচরণ বর্ণনা আছে। এই মেথডটি সেটটাইমাউট ফাংশন ব্যবহার করে 2.5 সেকেন্ডের পরে আনসাবস্ক্রাইব অবজেক্ট ফেরত দেয়।

এটি কাজ করার জন্য, আমাদের সাবস্ক্রিপশনটি উৎপন্ন করে দেওয়া “interval” অপারেটরটি অবশ্যই সংযোজন করতে হবে যা সাবস্ক্রাইব থেকে পরে অন-সাবস্ক্রাইব করা যাবে।

এখানে স্ট্যান্ডার্ড rxjs লাইব্রেরি ব্যবহার করে উদাহরণটি কাজ করেছে:

এখানে আমাদের কাস্টম rxjs লাইব্রেরির একটি ইনস্ট্যান্স আছে, এটি নিম্নলিখিত উদাহরণে উপস্থাপিত হয়েছে:

চমৎকার যে দুটি একই ফলাফল দিয়ে — এটি দেখায় কত আমাদের কাস্টম rxjs লাইব্রেরি বিস্তৃত হয়েছে। আমরা এখন সংখ্যা যানজট পাঠানোর জন্য interval অপারেটরটি ব্যবহার করতে পারি এবং প্রয়োজন হলে অন-সাবস্ক্রাইব করতে পারি।

সংকলনে, স্ক্র্যাচ থেকে কাস্টম Observable লাইব্রেরি তৈরি করা হলো যা অবসরপ্রাপ্ত তথ্য ঝরে নিয়ে সমস্ত অসমস্যার জন্য ব্যবহৃত হতে পারে। কয়েকটি লাইন কোড দিয়েই আমরা বেসিক Observable কার্যকারিতা অর্জন করতে পেরেছি এবং প্রয়োজনীয়তা অনুযায়ী এটি সুষ্ঠুভাবে উন্নয়ন করতে পারি।

Observables এর সৌন্দর্য হলো তাদের সফলভাবে ব্যবহার ও পরিবর্তনশীলতা। এগুলি সহজতর এবং এপ্লিকেশনগুলিতে সিম্পল ডেটা ইমিশন থেকে শুরু করে কমপ্লেক্স নেটওয়ার্ক রিকুয়েস্ট পর্যন্ত যেকোনো ধরনের ডেটা স্ট্রিম হ্যান্ডেল করতে ব্যবহৃত হতে পারে। অবজারভেবল কাজ করার নিয়ম বুঝে নিজেকে প্রশিক্ষিত করে নতুন করে ওবজারভেবল লাইব্রেরী তৈরি করা একটি উত্তম অনুশীলন। কয়েকটি কোড লাইন দিয়ে আমরা বেসিক অবজারভেবল ফাংশনালিটি সম্পন্ন করতে পারি এবং এটি মূলত সমস্ত ব্যবহারের জন্য উন্নয়নযোগ্য করে নেওয়া যায়।

পড়ার জন্য ধন্যবাদ এবং আশা করি এই নিবন্ধটি আপনাকে অবজার্ভেবল এবং নিজের কাস্টম অবজার্ভেবল লাইব্রেরি তৈরি করতে কিভাবে সম্পূর্ণ করতে সাহায্য করেছে। কয়েকটি লাইন কোড দিয়ে, আমরা বেসিক অবজার্ভেবল ফাংশনালিটি অর্জন করতে পেরেছি এবং এটি সম্ভবত সবচেয়ে বেশি ব্যবহৃত মামলাগুলির জন্য অবজার্ভেবল লাইব্রেরি পুরোপুরি RxJS প্যাকেজ সহ আমাদের প্রকল্পে যুক্ত করার প্রয়োজন নেই।

চেক আউট মাই ওপেন সোর্স ফ্রেমওয়ার্ক QQ:

https://github.com/jsmuster/qq

আমার অন্যান্য নিবন্ধগুলি দেখুন।

• Angular Routing in 5 Minutes
• Angular Injector In 5 Minutes
• Searching for a good JavaScript Developer — the Interview Questions

দয়া করে এই নিবন্ধটির কিছু 👏 দিয়ে এবং শেয়ার করুন!.

Dans cet article, j’explorerai le concept d’Observables en créant une petite bibliothèque à partir de zéro. À travers cet exercice, nous plongerons dans le fonctionnement interne des Observables, comment ils peuvent être créés et comment ils peuvent être utilisés pour gérer les flux de données asynchrones.

Alors, comment pouvons-nous créer des Observables à partir de zéro ? Jetons un coup d’œil à la documentation RxJS pour une recette, comme vous le voyez, une classe Observable étend une classe Subscribable, et Subscribable est une interface avec la méthode subscribe, donc notre Observable devrait avoir une méthode subscribe, n’est-ce pas ?

Le constructeur est un autre aspect important, il a un paramètre facultatif, qui est une fonction qui est appelée lorsque l’Observable est abonné.

Cette fonction reçoit un objet abonné de type Observer, qui peut être utilisé pour émettre de nouvelles valeurs en utilisant la méthode next, ou signaler une erreur en utilisant la méthode error, ou indiquer une fin réussie en utilisant la méthode complete.

Sur la base des informations ci-dessus, l’Observable devrait fonctionner comme ceci :

let obs = rxjs.Observable.create((subscriber) => {
  
  subscriber.next(1);
  subscriber.next(2);
  subscriber.next(3);

subscriber.complete();
});

obs.subscribe((value) => {
  console.log(value);
});

1. Créer l’Observable et passer une méthode dans le constructeur qui dispatchera les valeurs dans l’abonné de l’Observable.
2. Enfin, abonnez-vous à l’Observable en passant une méthode qui recevra les valeurs.

La méthode subscribe invoque la fonction de constructeur passée à l’Observable. Cela signifie que s’abonner à un Observable est le moment où il commence à exécuter sa logique, plutôt que pendant la création, comme on le croit généralement. La méthode constructeur est sauvegardée en tant que référence et attend que la méthode subscribe l’exécute.

Dans mon exemple, je passe une fonction dans le constructeur de l’Observable, qui est exécutée immédiatement lorsque nous nous abonnons, MAIS elle est généralement fournie par une implémentation de bibliothèque qui spécifie ce que l’Observable émettra et quand il l’émettra. Et c’est souvent asynchrone.

Par exemple, dans un scénario du monde réel, le service suivant exécutera la méthode subscribe de manière asynchrone, uniquement lorsque des données sont reçues avec succès à partir d’un service HTTP.

getConfigFromServer() {

this.configService.getConfig()
    .subscribe((data: Config) => this.config = {
        heroesUrl: data.heroesUrl,
        textfile:  data.textfile,
        date: data.date,
    });

}

ou voici un Observable qui émettra une valeur toutes les secondes après que nous nous y sommes abonnés :

var observable = rxjs.Observable.create((subscriber) =>{
  let count = 0;
  let interval;
         
   interval = setInterval(()=>{
        subscriber.next('Hello world (' + count++ + ')');
        
        if(count > 5)
        {
         clearInterval(interval);
          
          subscriber.complete();
    }
   }, 1000);

});

Maintenant, implémentons l’Observable et exécutons enfin ces exemples de code.

Voici le code d’exemple utilisant la bibliothèque rxjs pour l’exécuter :
https://jsfiddle.net/jsmuster/tgkc05vy/

Et voici ma version avec une implémentation Observable personnalisée :
https://jsfiddle.net/jsmuster/u39t6bh8/

Les deux fonctionnent de la même manière. Nous obtenons le journal de console suivant dans les deux exemples :

Maintenant, regardons l’exemple où Observable émet une valeur toutes les secondes :

Nous obtenons la sortie de console suivante :

Avec seulement 80 lignes de code, nous avons été en mesure d’obtenir une fonctionnalité Observable de base et tout semble fonctionner parfaitement. Fantastique !

Amusons-nous un peu et améliorons la fonctionnalité ! Nous pouvons créer une version allégée de la bibliothèque Observables qui répond à la plupart de nos cas d’utilisation, sans avoir à inclure l’intégralité du package RxJS dans nos projets.

Le code suivant fait partie de la documentation RxJS. Pour que cela fonctionne, nous devons ajouter la méthode “of”, comme le montre l’exemple suivant de la documentation RxJS.

J’ai implémenté l’opérateur “of” dans la version mise à jour, et j’ai apporté des modifications à la façon dont les méthodes sont exécutées dans l’Observable. Essentiellement, je les appelle via une méthode d’enveloppe via un “appel” pour m’assurer que l’objet abonné est “this” dans les méthodes next, complete et error. Jetez un coup d’œil à l’exemple suivant :

sub.complete = () => {
  subscriber.complete.call(subscriber);
};

Vous devriez obtenir le journal de console suivant lors de son exécution :

Sur la page RxJS Observable, il y a une autre illustration de comportement qui utilise la méthode “interval”. Cette méthode renvoie un objet d’abonnement qui est désabonné après 2,5 secondes en utilisant la fonction setTimeout.

Pour que cela fonctionne, nous devons inclure l’opérateur “interval” qui génère l’abonnement, que nous pouvons ensuite désabonner.

Voici l’exemple qui fonctionne en utilisant la bibliothèque RxJS standard :

Ici, nous avons une instance de notre bibliothèque RxJS personnalisée en action, comme le montre cet exemple :

C’est incroyable que les deux produisent une sortie — cela démontre à quel point notre bibliothèque RxJS personnalisée s’est étendue. Nous pouvons maintenant utiliser l’opérateur interval pour envoyer des nombres à travers l’Observable et nous désabonner lorsque cela est nécessaire.

En conclusion, la création d’une bibliothèque Observable personnalisée à partir de zéro est un excellent exercice pour comprendre le fonctionnement interne des Observables et comment ils gèrent les flux de données asynchrones. Avec seulement quelques lignes de code, nous avons pu obtenir une fonctionnalité Observable de base et l’améliorer pour répondre à la plupart des cas d’utilisation.

La beauté des Observables réside dans leur flexibilité et leur facilité d’utilisation. Ils peuvent être utilisés pour gérer des flux de données dans une variété de scénarios, de simples émissions de données à des demandes réseau complexes. En comprenant le fonctionnement des Observables, nous pouvons débloquer un tout nouveau niveau de fonctionnalité dans nos applications.

Merci de lire et j’espère que cet article vous a donné une meilleure compréhension des Observables et comment créer votre propre bibliothèque Observable personnalisée.

Découvrez mon framework open source QQ :

https://github.com/jsmuster/qq

Découvrez mes autres articles :

• Angular Routing in 5 Minutes
• Angular Injector In 5 Minutes
• Searching for a good JavaScript Developer — the Interview Questions

Veuillez donner à l’article quelques 👏 et le partager !

Entonces, ¿cómo creamos Observables desde cero? Echemos un vistazo a la documentación de RxJS para obtener una receta. Como verás, la clase Observable extiende un Subscribable, y Subscribable es una interfaz con el método subscribe, por lo que nuestro Observable debería tener un método subscribe, ¿verdad?

El constructor es otro aspecto importante, tiene un parámetro opcional que es una función que se llama cuando el Observable se suscribe.

Esta función recibe un objeto suscriptor del tipo Observer, que se puede utilizar para emitir nuevos valores utilizando el método next, o para señalar un error utilizando el método error, o para indicar la finalización exitosa utilizando el método complete.

Basándonos en la información anterior, el Observable debería funcionar de la siguiente manera:

let obs = rxjs.Observable.create((subscriber) => {
  
  subscriber.next(1);
  subscriber.next(2);
  subscriber.next(3);

subscriber.complete();
});

obs.subscribe((value) => {
  console.log(value);
});

Crear el Observable y pasar un método al constructor que despachará los valores al suscriptor del observable.

Finalmente, suscríbete al Observable pasando un método que recibirá los valores.

El método subscribe invoca la función del constructor pasada al Observable. Lo que significa que suscribirse a un Observable es el punto en el que comienza a ejecutar su lógica, en lugar de durante la creación, como comúnmente se cree. El método constructor se guarda como una referencia y espera que se ejecute el método subscribe.

En mi ejemplo, estoy pasando una función al constructor del Observable, que se ejecuta de inmediato cuando nos suscribimos, PERO generalmente es proporcionada por una implementación de biblioteca que especifica qué emitirá el Observable y cuándo lo emitirá. Y a menudo es asíncrono.

Entonces, por ejemplo, en un escenario del mundo real, el siguiente servicio ejecutará el método subscribe de manera asíncrona, solo cuando se reciban datos de un servicio HTTP correctamente.

getConfigFromServer() {

this.configService.getConfig()
    .subscribe((data: Config) => this.config = {
        heroesUrl: data.heroesUrl,
        textfile:  data.textfile,
        date: data.date,
    });

}

o aquí hay un Observable que emitirá un valor cada segundo después de que nos suscribamos a él:

var observable = rxjs.Observable.create((subscriber) =>{
  let count = 0;
  let interval;
         
   interval = setInterval(()=>{
        subscriber.next('Hello world (' + count++ + ')');
        
        if(count > 5)
        {
         clearInterval(interval);
          
          subscriber.complete();
    }
   }, 1000);

});

Ahora, implementemos el Observable y finalmente ejecutemos estos ejemplos de código.

Aquí está el código de muestra que usa la biblioteca rxjs para ejecutarlo:
https://jsfiddle.net/jsmuster/tgkc05vy/

Y aquí está mi versión con implementación personalizada de Observable:
https://jsfiddle.net/jsmuster/u39t6bh8/

Ambos funcionan igual. Obtenemos el siguiente registro en la consola en ambos ejemplos:

Ahora, echemos un vistazo al ejemplo en el que el Observable emite un valor cada segundo:

Obtenemos la siguiente salida en la consola:

Con solo 80 líneas de código, pudimos lograr una funcionalidad básica de Observable y todo parece funcionar perfectamente. ¡Fantástico!

¡Divirtámonos y mejoremos la funcionalidad! Podemos crear una versión simplificada de la biblioteca Observables que satisfaga la mayoría de nuestros casos de uso, sin tener que incluir el paquete completo de RxJS en nuestros proyectos.

El siguiente fragmento de código es parte de la documentación de RxJS, para hacer que funcione, necesitamos agregar el método “of”, como se muestra en el siguiente ejemplo de la documentación de RxJS.

He implementado el operador “of” en la versión actualizada y he hecho algunas modificaciones en cómo se ejecutan los métodos dentro del Observable. Esencialmente, los llamo a través de un método envoltorio a través de una “llamada” para asegurarme de que el objeto suscriptor sea “this” dentro de los métodos next, complete y error. Echa un vistazo al siguiente ejemplo:

sub.complete = () => {
  subscriber.complete.call(subscriber);
};

Deberías obtener el siguiente registro en la consola cuando lo ejecutes:

En la página de Observables de RxJS, hay otra ilustración de comportamiento que emplea el método “interval”. Este método devuelve un objeto de suscripción que se cancela después de 2.5 segundos utilizando la función setTimeout.

Para que esto funcione, debemos incluir el operador de intervalo que genera la suscripción, que luego podemos cancelar.

Aquí está el ejemplo funcionando usando la biblioteca estándar de RxJS:

Aquí tenemos una instancia de nuestra biblioteca RxJS personalizada en acción, como se demuestra en este ejemplo:

Es sorprendente que ambos produzcan salida, esto demuestra cuánto se ha expandido nuestra biblioteca RxJS personalizada. Ahora podemos utilizar el operador de intervalo para enviar números a través del Observable y cancelar la suscripción cuando sea necesario.

En conclusión, crear una biblioteca personalizada de Observable desde cero es un gran ejercicio para comprender el funcionamiento interno de los Observables y cómo manejan flujos de datos asincrónicos. Con solo unas pocas líneas de código, pudimos lograr una funcionalidad básica de Observable y mejorarla para satisfacer la mayoría de los casos de uso.

La belleza de los Observables radica en su flexibilidad y facilidad de uso. Se pueden utilizar para manejar flujos de datos en una variedad de escenarios, desde emisiones simples de datos hasta solicitudes de red complejas. Al comprender cómo funcionan los Observables, podemos desbloquear un nuevo nivel de funcionalidad en nuestras aplicaciones.

Gracias por leer y espero que este artículo te haya dado una mejor comprensión de los Observables y cómo crear tu propia biblioteca personalizada de Observable.

Echa un vistazo a mi marco de código abierto QQ:

https://github.com/jsmuster/qq

Echa un vistazo a mis otros artículos.

• Angular Routing in 5 Minutes
• Angular Injector In 5 Minutes
• Searching for a good JavaScript Developer — the Interview Questions

¡Por favor, dale al artículo algunos 👏 y compártelo!