<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Arslan Ali on Medium]]></title>
        <description><![CDATA[Stories by Arslan Ali on Medium]]></description>
        <link>https://medium.com/@arslanwrites?source=rss-9e818bfa5a82------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*OTXIxr_I6Gc2mFOCq3FzMA.png</url>
            <title>Stories by Arslan Ali on Medium</title>
            <link>https://medium.com/@arslanwrites?source=rss-9e818bfa5a82------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 18:18:05 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@arslanwrites/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Achieving Kubestronaut in 40 at 40]]></title>
            <link>https://arslanwrites.medium.com/achieving-kubestronaut-in-40-at-40-4ca86245d9f4?source=rss-9e818bfa5a82------2</link>
            <guid isPermaLink="false">https://medium.com/p/4ca86245d9f4</guid>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[growth-hacking]]></category>
            <category><![CDATA[kubestronaut]]></category>
            <category><![CDATA[certified-kubernetes]]></category>
            <category><![CDATA[the-linux-foundation]]></category>
            <dc:creator><![CDATA[Arslan Ali]]></dc:creator>
            <pubDate>Fri, 31 Jan 2025 09:21:19 GMT</pubDate>
            <atom:updated>2025-02-19T06:49:16.488Z</atom:updated>
            <content:encoded><![CDATA[<h3>Achieving Kubestronaut in 40 Days</h3><h4>I make the point that with experience it becomes easy to learn new technologies. Pardon my language and grammar as I prefer writing with natural intelligence no AI used.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/938/1*xwTk9ZnuCVZ7XrQkXbqYwQ.png" /></figure><p>My first interaction with Kubernetes was in Fall of 2017, when I planned to move my Meteorjs Application away from Heruku as I wanted to implement microservices to scale individual service units.</p><p>After 3 years of playing around with Kubernetes, in 2020, a recruiter wanted to discuss further a consulting requirement and said “It would have been great if you had a Kubernetes Certification”. I told him to schedule an interview with the client and I will ensure he will have it before that. Indeed I passed it in less than a week but with a score of 71% only. Scores were not important to me then, I got the offer letter but the offer was pulled with a regret letter because of the 2020 Pandemic.</p><p>Fast forward to December 2024, our company <a href="https://lowcodesol.com/">Low Code Solutions</a> got a Lead to set up an on-prem capability of Openshift Multi-Cluster for a Telco. This is when I planned to support the company’s Pre-Qualification in front of the Customer and started preparing for Kubestronaut Training on Dec 13 2024.</p><p>Due to my busy schedule, I was not able to spend more than a couple of hours a day, which is why I am going to share how smartly it will be possible for you to achieve your certifications in even less time. I will discuss each one in the sequence that I attempted:</p><h3>CKA</h3><p>Starting off with CKA, as I thought it would have been the hardest, however, it turned out to be the easier one compared to CKAD and CKS. If you have a reasonable knowledge of Linux basics, Docker and Linux Package Managers then you can prepare for it in less than a week.</p><p><strong>Important Topics:</strong><br>Following are the most important topics, where you will need to spend most of your time:</p><ol><li>Imperative Commands to create, delete and update Kubernetes Resources, e.g: Pods, Deployments, Services etc.</li><li>Cluster Installation, Upgrade and Basic Troubleshooting</li><li>Understanding of Kubernetes Services Architecture (ETCD, Kube-Scheduler, Kube-Controller-Manager, Kube-ApiServer, etc.)</li><li>Understanding of Deployment, and Management of Nodes into a Cluster, understanding the role of Kubelet and Kube-Proxy</li><li>Understanding Static Pods and Important Config file Locations e.g: /etc/kubernetes, /var/lib/kubelet etc.</li><li>Understanding of TLS and Certificate Management.</li><li>Backup and Restore of ETCD</li><li>RBAC, how to create roles, rolebindings, clusterroles, clusterrolebindings, and service accounts. If you understand these 5 elements in depth then RBAC will be a piece of cake for you. (tip: it will also help you with your CKS exam, so spend more time with it.)</li></ol><h3>CKAD</h3><p>CKAD is more about deploying and managing cloud-native applications on Kubernetes. Remember if you have done CKA, then it will help you in preparing for CKAD, and even for CKS, all you need to do is prepare some additional concepts, here are the important topics that I think you would need to emphasize more than others:</p><ol><li>Imperative Commands as mentioned for CKA also.</li></ol><pre>k create --help <br>k run --help<br>k expose --help<br>k edit --help<br>k replace -f file.yaml -n namespace<br>k delete --help<br>k scale --help</pre><p>2. Understand the concept of Ingress Resource and practice the following command:</p><pre>k create ingress NAME --rule=host/path=service:port[,tls[=secret]]</pre><p>3. Practice the creation, updation and mounting of secrets, configmaps, service accounts, volumes and volume claims in Pods/Containers.</p><p>4. Taints, Tolerations, Node Affinity, Node Selector and nodeName for controlling how we schedule pods on specific nodes. Also understand why we need node affinity when we have taints and tolerations.</p><p>5. Init and Sidecar Containers</p><p>6. Logging, Monitoring and Probes.</p><p>7. Green-Blue and Canary Deployments</p><p>8. Pod SecurityContext and RBAC as mentioned above also.</p><p>9. Custom Resource Definitions and HELM</p><h3>CKS</h3><p>The CKS exam is where the real challenge begins. If you excelled in the CKA and CKAD exams, it’s easy to underestimate the difficulty of the CKS. I made the same mistake — despite my extensive Kubernetes experience and proficiency with the complex commands and vim, I failed on my first attempt. So before I tell you what to prepare most, let me share the mistakes:</p><p><strong>Mistake #1:</strong><br>I spent my first hour on just 3 questions, which left me with no chance to attempt all the questions and I was not able to even view the last 3 questions. So time management is important. Cluster Upgrade, Setting Up Audit Policy, and ImagePolicyWebhook are the most time-consuming questions, attempt them wisely.</p><p><strong>Mistake #2:</strong><br>I had clear and vivid core concepts of all the topics, so I thought I will take help from the documentation and solve all the questions. But Practice is the key, you can’t switch between the documentation and expect that you will finish in time.</p><p><strong>Mistake #3:<br></strong>I did not practice troubleshooting cluster crashes. So practice troubleshooting Api-Server or Kubelet issues by utilizing docker/crictl logs, displaying logs from /var/log folders for pods and containers, and other interactive ways. There are specific scenarios of such troubleshooting in <a href="https://killercoda.com/">KillerCoda Playgrounds</a>.</p><p><strong>CKS Exam TIPs:</strong></p><ol><li>Falco questions are tricky, most of my peers were unable to solve it, so do not spend too much time on it.</li><li>Make sure you know the difference between Layer 3, Layer 4, and Layer 7 Cillium Policies.</li><li>Some questions have supplementary tasks at the end of the questions. Read the whole scenario, do not assume that you are done.</li><li>Verify your answers. For every scenario, practice verification process. Spare at least 15 to 20 min for your answer verification, which leave you with 90 to100min for solving the scenarios.</li></ol><p><strong>Important Topics for CKS which requires extreme practice:</strong></p><ol><li>BOM, Benchmarking and Vulnerability scanning CLI Tools e.g: BOM, Kube-bench, Trivy, lsof, strace, netstat -plnt etc.</li><li>Api-Server Audit Log, ImagePolicyWebhook, Cluster Upgrade, Securing and Encrypting ETCD data and Setting up Network Policy (both native and cilium)</li><li>Sandboxing the Containers, Immutability and Runtime Security with Falco etc.</li><li>Setting proper Pod Security Context, AppArmor/Seccomp profiles, readOnlyRootFilesystem, allowPrivilegeEsclation etc. in context of Pod Security Standards.</li><li>Ingress Resource with TLS Secrets and its important annotations. (Hint: Imperitive Commands will help save time)</li></ol><p><strong>General Exam Tips:</strong></p><ol><li>Practice is the key, you may know and understand the concepts in depth, but remember CKA, CKAD and CKS are performance-based exams, you will get 16 to 18 scenarios to solve. In our consulting world, we take at least a day or two to solve even one.</li><li>Solve at least 2 to 3 times the <a href="https://killercoda.com/">KillerCoda Playgrounds</a> thanks to <a href="https://www.linkedin.com/in/kimwuestkamp/">Kim Wustkamp</a>, the founder, for keeping it free. However, I would recommend taking at least a 1 month pro subscription because this way you will get the exam desktop which will help you familiarize yourself with the exam environment. I wasted 5 minutes finding how to do basic functionalities like copy/pasting and keeping notes etc.</li><li>Get a <a href="https://kodekloud.com/">KodeKloud</a> Subscription, it is a gold mine of learning material for Infrastructure and DevOps. If you are validating</li><li>VIM proficiency: you need to start using some shortcuts in vim, some of the most handy are the ones which will help you with: Copy, Past, Delete Line, replace-in-place, Indentation, set number to show line numbers, visual tool to copy/duplicate multiple lines, find text.</li><li>Grep proficiency: Grep can help you in unimaginable ways e.g: finding a particular vulnerability in a BOM scan, finding a file containing a specific text in a folder of multiple such files (hint: grep -r), greping two options (hint: grep -E “one|two”).</li></ol><p><strong>Profession Tips:</strong></p><p>Even though <a href="https://www.linuxfoundation.org/">The Linux Foundation</a> and the <a href="https://www.cncf.io/">CNCF</a> has done a great job at creating and crafting these exams and certifications programs where they have made sure that only the exceptional comes out shining. However, it is important to know that real world scenarios are even more complex and worse than the ones you will encounter in these exams. So, getting the certification does not guarantee that you are suitable for some challenging role. The certification will help you to land an interview though, but to get a great offer you will need a lot more than the certifications. So after hiring 100s of individual I am going to share some of the most important traits that you also need to land your dream job:</p><ol><li>Prepare for these certifications keeping your professional goals in mind. Prepare yourself for the industry challenges not only for the exam, have your previous job scenarios in mind and relate every topic with your current or upcoming role in business.</li><li>Work on your soft-skills, make sure you know how to express your learning and experience effectively. Understand and make use of industry and technology buzz words and their concepts e.g: cloud-native, cluster-hardening, encryption at rest, zero-trust, software security compliance and standards etc.</li><li>Make yourself vocal on platform like <a href="https://linkedin.com/">LinkedIn</a> and <a href="https://x.com/">X</a> by sharing your thoughts on generic technology discussions. (even I am struggling to do that often)</li><li>Express your ideas in Diagrams and Visuals. Make every presentation or discussion more meaningfully with your visual skills.</li><li>Learn how to explain a complex idea or architecture to multiple audiences, tech, non-tech, business or a layman.</li><li>Write blogs/articles while your are learning or even after accomplishing a goal. Focus on spreading the knowledge, tips and tricks rather than publicity or marketing.</li><li>Start Contributing in Kubernetes and/or other CNCF Projects. Start by attending the weekly or monthly meetings and help yourself in understanding the working group processes until you think you are ready to take on the challenge of taking an assignment.</li><li>Remember the motive, we are working to help make the world a better place.</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4ca86245d9f4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Common security vulnerabilities in Kubernetes Clusters]]></title>
            <link>https://arslanwrites.medium.com/common-security-vulnerabilities-in-kubernetes-clusters-aa9903e65d53?source=rss-9e818bfa5a82------2</link>
            <guid isPermaLink="false">https://medium.com/p/aa9903e65d53</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[kubernetes-security]]></category>
            <category><![CDATA[dev-sec]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[cloud-security]]></category>
            <dc:creator><![CDATA[Arslan Ali]]></dc:creator>
            <pubDate>Thu, 20 Oct 2022 16:43:05 GMT</pubDate>
            <atom:updated>2022-10-20T16:43:05.462Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Lxa9Jb4oTf4M-7fb" /><figcaption>Photo by <a href="https://unsplash.com/@freeche?utm_source=medium&amp;utm_medium=referral">Kvistholt Photography</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>54% of enterprises mentioned Security as their biggest challenge in Kubernetes. Almost 7 out of 10 have limited or lack resources to manage security as Kubernetes has a steep learning curve. Now it is not only the Kubernetes lacking native security scanning ability rather any vulnerability in an application running on the cluster may lead to a bigger threat. Here are a few hacks from the past:</p><ul><li><a href="https://www.bleepingcomputer.com/news/security/over-900-000-kubernetes-instances-found-exposed-online/">Over 900,000 Kubernetes Instances are found Exposed Online</a></li><li><a href="https://www.wired.com/story/cryptojacking-tesla-amazon-cloud/">Tesla Clusters on Amazon Cloud were Hacked with Cryptojackers</a></li><li><a href="https://sysdig.com/blog/exposed-prometheus-exploit-kubernetes-kubeconeu/">Exposed Prometheus exploit Kubecon EU</a></li></ul><p>Now that we know that a single container vulnerability may compromise the overall security of the cluster, let us review a few aspects.</p><h3>Public Nodes</h3><p>A public cluster is one whose Nodes are accessible from the internet, technically they have one or more networks configured with ExternalIP. You can test it using the following command</p><pre>kubectl get node &lt;node-name&gt; -o jsonpath=&#39;{.status.addresses}&#39; | grep ExternalIP</pre><p>Replace &lt;node-name&gt; with your node name in the cluster.</p><p>If any of your nodes have an ExternalIP then it is exposed, which means all your containers/pods running on this node are also exposed to the public. It is highly recommended to keep your nodes private and route the outgoing traffic using a <a href="https://cloud.google.com/nat/docs/overview">Cloud NAT</a>.</p><h3>Public Kubernetes Dashboard</h3><p>Kubernetes Dashboard is an open-source deployment used to manage and monitor cluster deployments and other objects. It is available on all clusters and can be deployed or installed with a few simple commands. Here is how you can also deploy it on your cluster:</p><pre>kubectl apply -f <a href="https://raw.githubusercontent.com/kubernetes/dashboard/v2.6.1/aio/deploy/recommended.yaml">https://raw.githubusercontent.com/kubernetes/dashboard/v2.6.1/aio/deploy/recommended.yaml</a></pre><p>Once installed you can proxy it to access it from your remote machine (machine with cluster access) using kubectl proxy command. Now you can access your dashboard using the following URL: <a href="http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/">http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/</a></p><blockquote><em>Note: The kubeconfig authentication method does not support external identity providers or X.509 certificate-based authentication. The UI can only be accessed from the machine where the command is executed. See </em><em>kubectl proxy --help for more options.</em></blockquote><p>Now, this is your standard access method, if one of your vulnerable (or compromised) containers has the access to your dashboard then the hacker will have access to the whole Kubernetes cluster.</p><h3>Exposed Observability Platform</h3><p>If you are using Prometheus, Grafana, or Splunk, chances are that you are exposing them by using an external load balancer. These and other observability platforms are often used to monitor and prevent performance, security, and other issues in a cluster. Once exposed to public traffic either through an ingress or through a vulnerable (or compromised) container may lead to a brute force attack. A hacker can take control of the whole cluster once getting access to such a platform.</p><h3>Privileged Container</h3><p>A privileged container is one that has full access to itself and through the <strong>reverse shell</strong> technique it can also get full control over its encapsulating Node. Full access to a node exposes all the images stored on it and it may lead to access to other nodes in the network exposing all available resource objects on the cluster. Hackers may deploy a cron job or hijack the resources of the nodes without even letting it be noticed from the Kubernetes platform.</p><h3>Conclusion</h3><p>I have just scratched the surface and mentioned some of the most common security vulnerabilities. An attacker can use impact techniques to destroy, abuse, or disrupt the normal behavior of an environment. If you would like to read more on this topic then read the <a href="https://blog.arslanali.io/series/secops-kubernetes">full series on Kubernetes security</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=aa9903e65d53" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How AI can help remediate Kubernetes Vulnerabilities]]></title>
            <link>https://arslanwrites.medium.com/how-ai-can-remediate-kubernetes-security-d19a75e53b02?source=rss-9e818bfa5a82------2</link>
            <guid isPermaLink="false">https://medium.com/p/d19a75e53b02</guid>
            <category><![CDATA[devsecops]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[devsecops-tool]]></category>
            <category><![CDATA[cybersecurity]]></category>
            <dc:creator><![CDATA[Arslan Ali]]></dc:creator>
            <pubDate>Thu, 20 Oct 2022 14:44:50 GMT</pubDate>
            <atom:updated>2022-10-20T15:01:43.272Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="How AI can remediate kubernetes" src="https://cdn-images-1.medium.com/max/1024/0*qX_sSx_3RHtQ89TC" /><figcaption>Photo by <a href="https://unsplash.com/@markusspiske?utm_source=medium&amp;utm_medium=referral">Markus Spiske</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>More than half of all enterprises consider <strong>security as their biggest</strong> challenge when publishing their microservice workloads in production. 50% require developers to <strong>use validated images</strong> only, around 80% want to have a <strong>DevSecOps initiative</strong>, more than 40% <strong>consider DevOps</strong> as the role most responsible for Kubernetes security, and most importantly, more than half have delayed application deployment due to security concerns.</p><blockquote><em>According to the </em><strong><em>State of Kubernetes Security Report 2022</em></strong><em>, security is one of the biggest concerns with container adoption, and security issues continue to cause delays in deploying applications into production.</em></blockquote><p>In the last 1 year, 93% said that they have experienced at least one major security incident. More than 30% of them have experienced customer or revenue losses due to these incidents. According to a recent study, 95% of the breaches were due to human errors.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EbmAsopY7RaCx-sFjUgdKQ.png" /><figcaption>Source (Red Hat State of Kubernetes Security Report — 2022)</figcaption></figure><h3>Impact</h3><p><strong>1. Data Compromise</strong><br>An attacker with access to business or infrastructure data can leak or destroy the data.</p><p><strong>2. Resource Hijacking</strong><br>If an attacker gets access to a node, or any compute resource in a cluster; he can easily run resource-hungry scripts like crypto mining (crypto-jacking), AI model processing, etc.</p><p><strong>3. Denial of Service</strong><br>DoS can be achieved using buffer overflow (by flooding general requests), ICMP flooding (by sending spoofed packets), or SYN flooding (by sending false requests to the server without a handshake). DoS attack is meant to shut down a computer or make a network inaccessible.</p><p><strong>4. Ransom</strong><br>An attacker can take over the Management Layer or even remove the (important) data and ask for a ransom in return for the data or control.</p><p><strong>5. Loss in Customers and/or revenue</strong><br>One may incur major losses once a service is down because of any reason mentioned above.</p><h3>AIOps to the Rescue</h3><p>KaiOps is an AI/ML SaaS-based tool, that connects with your clusters using a secure connection with the KubeAPI server. You may need to update the default security profile to enable the KaiOps security agents to scan and flag different vulnerabilities as low, medium, or high on threats. Once a threat is found, it will send you notifications on your preferred communication channels and inform you of the possible implication, remediation, and/or prevention strategies.</p><p>KaiOps scans the cluster every few seconds at runtime and observes the most important Kube events related to network, storage, and workload. It also watches all whitelisted Kube objects for any change and runs AI inferences using GNN (Graph Neural network) to detect misconfigurations, policy violations, and potential threats.<br>Cluster monitoring methods are divided into 10 classes:</p><p><strong>1. Access</strong><br>Exposing cluster Nodes publicly may give the same access to containers with potential vulnerabilities and may lead to attackers penetrating the cluster.<br><strong>2. Execution</strong><br>Execution of commands is monitored very closely as an SSH server or a bash script running in a container can be compromised using brute force attacks.<br><strong>3. Persistence</strong><br>Writeable hostPaths in containers or persistence volumes, backdoor containers, or exposed/compromised CronJobs are also monitored to reduce the risk of penetration.<br><strong>4. Privilege</strong><br>The security contexts of all running containers are continuously monitored along with the RBAC context.<br><strong>5. Defense Evasion</strong><br>Kube objects deletion, clearing of container logs, or connections through proxy servers are some of the defense evasive events its monitors according to security policy.<br><strong>6. Credentials Leak</strong><br>Leakage of Kube CA data of an exposed ApiServer or cloud credential files on a hosted Kubernetes service. KaiOps continuously monitors Kube’s internal network and even the object calls data, which is fed into the GNN for AI model training for anomaly detection.<br><strong>7. Discovery</strong><br>Exposed Observability platforms, K8s dashboard, ApiServer, and similar other services may lead an attacker to penetrate the cluster.<br><strong>8. Lateral Movement</strong><br>Writeable volumes mounted on hosts, exposed access to cloud resources, privileged service accounts assigned to a container, exposed application configuration through environment variables, core DNS poisoning, arp poisoning, IP spoofing, or even public IPs assigned to a container may compromise the security of a cluster. KaiOps flags its vulnerabilities and whitelists some of them based on a user-defined security policy.<br><strong>9. Collection</strong><br>Images from unknown or public sources and compromised container images can jeopardize cluster security. KaiOps continuously scans for Image and Package Vulnerabilities using static image scanners and feed the data into the AI inference engine.<br><strong>10. Custom K-Tactics</strong><br>There are some custom patented K-Tactics that KaiOps uses to help its GNN Models detect errors, ambiguities, vulnerabilities, and even anomalies. These K-Tactics are also used to reduce nuisance alarms and notifications.</p><h3>Conclusion</h3><p>With almost <a href="https://www.bleepingcomputer.com/news/security/over-900-000-kubernetes-instances-found-exposed-online/"><strong>One Million</strong> Kubernetes clusters found to be exposed</a>, it is evident that most of the above security vulnerabilities are present in the vast majority of Kubernetes clusters, making remediation difficult. KaiOps uses its patented AI/ML agents by feeding the telemetry data into a GNN model to predict and detect security vulnerabilities, misconfigurations, and anomalies. KaiOps SaaS is free for a cluster with up to 3 nodes for 14 days, and even includes live support in setting up their service for your clusters. KaiOps currently provide support for Native Kubernetes Clusters on almost all hosted Kubernetes services. Use this <a href="https://portal.kaiops.io/login/register">link to register</a> and get an extended 1 month of a free trial.</p><h3>References</h3><ul><li><a href="https://www.redhat.com/en/resources/state-kubernetes-security-report">Red Had State of Security Report</a></li><li><a href="https://kaiops.io/articles/aiops-use-cases">AIOps use cases</a></li><li><a href="https://blog.arslanali.io/common-security-issues-in-kubernetes-part-1">Common Security Issues found in almost all Kubernetes Deployments</a></li><li><a href="https://www.weave.works/blog/mitre-attack-matrix-for-kubernetes">MITRE ATT&amp;CK Matrix</a></li><li><a href="https://www.youtube.com/watch?v=ka0C09CAfho">Kubernetes Security Workshop</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d19a75e53b02" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Application Deployment using Gitlab CI/CD on Managed Kubernetes Cluster at GCP]]></title>
            <link>https://faun.pub/application-deployment-using-gitlab-ci-cd-on-managed-kubernetes-cluster-at-gcp-72b59496979c?source=rss-9e818bfa5a82------2</link>
            <guid isPermaLink="false">https://medium.com/p/72b59496979c</guid>
            <category><![CDATA[gcp]]></category>
            <category><![CDATA[continuous-deployment]]></category>
            <category><![CDATA[continuous-integration]]></category>
            <category><![CDATA[gitlab]]></category>
            <category><![CDATA[kubernetes]]></category>
            <dc:creator><![CDATA[Arslan Ali]]></dc:creator>
            <pubDate>Tue, 10 Mar 2020 19:38:31 GMT</pubDate>
            <atom:updated>2020-03-24T16:43:45.452Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>Step by Step guide to deploy a <a href="https://github.com/dockersamples/example-voting-app">mono-repo of microservices</a> on GCP using Gitlab CI/CD Pipelines.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/646/1*dbK6DLeZiJAzj_87-z0fPA.png" /></figure><p>In continuation of my <a href="https://medium.com/@arslan_53972/microservices-based-application-deployment-using-gitlab-ci-cd-on-docker-swarm-at-digital-ocean-586eefb07294">previous article</a> where I’ve deployed the same mono-repo of microservices application on Docker Swarm instead of Kubernetes, we will repeat most of the steps and I will try to be brief about all the steps except the deployment configuration. So it is highly recommended that you read <a href="https://medium.com/@arslan_53972/microservices-based-application-deployment-using-gitlab-ci-cd-on-docker-swarm-at-digital-ocean-586eefb07294">this article</a> first.</p><h3>Pre-requisites</h3><p>You need to have billing enabled account on GCP (Google Cloud Platform) for creating and running a Kubernetes cluster.</p><blockquote><a href="https://gitlab.com/docker-microservice/ci-cd-example-voting-app-k8s">The complete code of this article resides here…</a></blockquote><h3><strong>Agenda</strong></h3><p>In this article, we will clone the Docker’s official sample <a href="https://github.com/dockersamples/example-voting-app.git">voting app microservices example</a> into a Gitlab repository and implement CI/CD on a Kubernetes Cluster. I have broken it down into the following key steps:</p><ol><li>Create a Repository in a Repository Group at Gitlab</li><li>Create a K8s Cluster on GCP using Gitlab Integration</li><li>Install Helm, Gitlab Runner, Cert-Manager and Ingress on K8s Cluster</li><li>Change docker-compose.ymlto add image names</li><li>Change kube-deployment.ymlto contain Gitlab Environment Variables</li><li>Create .gitlab-ci.ymlfor the project.</li></ol><h3>Create Repository</h3><p>Create the repository in your Gitlab account for <a href="https://github.com/dockersamples/example-voting-app.git">voting app microservices example</a> using the CI/CD for External Repo option tab. Details on how to create it can be found <a href="https://medium.com/@arslan_53972/microservices-based-application-deployment-using-gitlab-ci-cd-on-docker-swarm-at-digital-ocean-586eefb07294">here.</a></p><h3>Create Kubernetes Cluster on GCP</h3><p>Gitlab provides the simplest way to create K8s cluster on GCP by using OAuth to log in to your Google Cloud Platform account and then by providing a simple form to select your project and other configurations for the required cluster. Do go through the <a href="https://docs.gitlab.com/ee/user/project/clusters/add_remove_clusters.html#before-you-begin"><strong>Before you begin instructions</strong></a> for GCP integration with Gitlab.</p><p>Simply open your Gitlab Project -&gt; Operations -&gt; Kubernetes -&gt; Add Kubernetes Cluster and then select Google GKE. It will request you to login using your Google Account for GCP. Once done you will be required to fill the following:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*avyH3eUYZ0bk4tCH4G45OQ.png" /><figcaption>Logged In using Google OAuth</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gqjFskI5mhiU0ieQS8XTIw.png" /><figcaption>Fill the Required Cluster Configuration</figcaption></figure><p>I have used <strong>e2-small</strong> as we are going to deploy a very basic application, however, you can choose a better machine size based on your application requirements. I have already created a Project in GCP with <strong>billing</strong>, <strong>GKE</strong> and other required APIs enabled. If you don&#39;t have one then click the Google Cloud Platform link below the <strong>GCP Project</strong> field and create one. Do not check the <strong>Enable Cloud Run for Anthos</strong> for now, as I will create another post for Cloud Runs where we will use the power of Anthos to deploy Application on Hybrid Cloud.</p><p>Now since your K8s Cluster is ready and integrated with Gitlab, so let&#39;s start with configuring it.</p><h3>Install Helm, Gitlab Runner, Cert-Manager and Ingress on K8s Cluster</h3><p>We need Helm Tiler to install other applications on our Clusters, so start off with the Installation of Helm. Once done then we would need a Gitlab Runner to run our CI/CD pipelines, we can utilize a separate Gitlab Runner but running it inside K8s cluster will allow us to run concurrent jobs as the runner will autoscale to full fill the requirements. Next, we need Cert-Manager; it is a native <a href="https://kubernetes.io/">Kubernetes</a> certificate management controller. It can help with issuing certificates from a variety of sources, such as <a href="https://letsencrypt.org/">Let’s Encrypt</a>, <a href="https://www.vaultproject.io/">HashiCorp Vault</a>, <a href="https://www.venafi.com/">Venafi</a>, a simple signing key pair, or self-signed. It will ensure certificates are valid and up to date, and attempt to renew certificates at a configured time before expiry. At last, we would need an Ingress Controller to route outside traffic to the inside of our cluster.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZYqRVkzXBlVbrhRSM_toCw.png" /><figcaption>Installed Applications on K8s Cluster</figcaption></figure><p>Once the ingress is installed you will get the Ingress Endpoint IP address to copy it and use it to make the Base Domain for your cluster as follows 1.2.3.4.nip.io and then press Save Changes. If you have a base domain with wildcard DNS configured then please use it instead.</p><blockquote>Entering a Base Domain with wildcard DNS support is very important for the CICD to work properly. As the value of KUBE_INGRESS_BASE_DOMAIN will be null if not saved here.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7VyVTcgdN_ZsrQQcB0XtlA.png" /><figcaption>Assign Ingress Endpoint as Base Domain using nip.io service</figcaption></figure><blockquote>Note: Once you are done installing the Gitlab Runner on kubernetes cluster, then goto your Project Settings -&gt; CI/CD -&gt; Runner and Disable Shared Runners for this project.</blockquote><h3>Change docker-compose.yml to add image names</h3><p>Since its a mono-repo which means all microservices are part of a single repository, so to build docker images we will make use of docker-compose.yml. We will issue a single docker-compose build command and it will start building images for all microservices. However since we are going to store our Docker images on Gitlab provided Docker Registry so let&#39;s change the image field of vote, result and worker microservice in the following format:</p><pre>image: __CI_REGISTRY_IMAGE__/vote:latest</pre><blockquote><a href="https://gitlab.com/docker-microservice/ci-cd-example-voting-app-k8s/-/blob/0b94482c6dd60e529ceed07c39ca73f7d7fc2d5b/docker-compose.yml">Here is the full code of docker-compose.yml.</a></blockquote><p>This way we are declaring a variable __CI_REGISTRY_IMAGE__ which we will replace while building it in our build CI/CD stage.</p><h3>Change kube-deployment.yml to contain Gitlab Environment Variables</h3><p>Similar to the above step we need to make some variables for kube-deployment.yml too. kube-deployment.yml contains all the manifests for creating and scaling our microservices. Each microservice is pulled following important fields:</p><p><strong>Labels<br></strong>Labels help us to group resources e.g app: redis-__CI_PROJECT_NAME__ will help us address a particular Redis deployment even if we have multiple Redis deployments for different projects, similarly ref:__CI_ENVIRONMENT_SLUG__ will help us identify the deployments of different environments for example review-master and live for the production environment.</p><p><strong>Annotations<br></strong>We will create app.gitlab.com/app: __CI_PROJECT_PATH_SLUG__ and app.gitlab.com/env: __CI_ENVIRONMENT_SLUG__ annotations for all of the pod templates under deployments to help Gitlab identify the deployments and show the rollout progress in environments tab like below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SYWxcQ20QMTX2INgtftZIA.png" /><figcaption>Live environment rollout status</figcaption></figure><p><strong>Image<br></strong>We need to keep the image field completely dynamic i.e in our current scenario we need to build different images for each build, we want to tag images with the appropriate version number or even branch names. We don&#39;t want to manually change the kube-deployment.yml every time we push a branch or a tag. So we will make reconfigure the image field as follows:</p><pre>containers:<br>- name: result<br>  <strong>image: __CI_REGISTRY_IMAGE__/result:__VERSION__</strong><br>  ports:<br>  - containerPort: 80<br>    name: result</pre><blockquote><a href="https://gitlab.com/docker-microservice/ci-cd-example-voting-app-k8s/-/blob/master/kube-deployment.yml">Here is the full code of kube-deployment.yml.</a></blockquote><p><strong>Ingress Resource<br></strong>The ingress resource is missing in the kube-deployment.yml provided by Docker Sample. So let&#39;s create it as follows:</p><pre>---<br># Ingress Resource</pre><pre>apiVersion: networking.k8s.io/v1beta1<br>kind: Ingress<br>metadata:<br>  name: app-ingress<br>  annotations:<br>    kubernetes.io/tls-acme: &quot;true&quot;<br>    kubernetes.io/ingress.class: &quot;nginx&quot;<br>  labels:<br>    app: ingress-__CI_PROJECT_NAME__<br>    ref: __CI_ENVIRONMENT_SLUG__</pre><pre>spec:<br>  tls:<br>  - hosts:<br>    - __CI_ENVIRONMENT_SLUG__.result.104.154.109.250.nip.io<br>    - __CI_ENVIRONMENT_SLUG__.104.154.109.250.nip.io<br>    # the secret used here is an unsigned wildcard cert for demo      purposes<br>    # use your own or comment this out<br>    secretName: tls-some-wildcard-demo<br>  rules:<br>  - host: __CI_ENVIRONMENT_SLUG__.result.104.154.109.250.nip.io<br>    http:<br>     paths:<br>     - path: /<br>       backend:<br>         serviceName: result<br>         servicePort: 5001<br>  - host: __CI_ENVIRONMENT_SLUG__.104.154.109.250.nip.io<br>    http:<br>     paths:<br>     - path: /<br>       backend:<br>         serviceName: vote<br>         servicePort: 5000</pre><pre>---</pre><blockquote><a href="https://gitlab.com/docker-microservice/ci-cd-example-voting-app-k8s/-/blob/master/kube-deployment.yml">Here is the full code of kube-deployment.yml.</a></blockquote><p>An Ingress Resource is just another Kubernetes resources with <strong>apiVersion</strong>, <strong>kind</strong>, <strong>metadata</strong>, and <strong>spec</strong>. However, in the <strong>spec</strong> section, we have to specify if the ingress has tls certificates available in its tls section. This is where the <strong>Cert-Manager</strong> will take the provided secretName and create Certificate authorities and keys automatically and generate tls certificates using Lets Encrypt service. Next comes the routing rules, this is where we need to specify hosts for each of our front-end microservice. In our case both Vote and Result Microservices are front-end microservices, so we will provide Environment Slug (review / live) as part of our DNS Domain name. If it is unclear, then please note that we will investigate environments more deeply when we will deploy our application.</p><h3>Create .gitlab-ci.yml for the project</h3><p>Now since we have set up everything, let’s start with our .gitlab-ci.yml , we will start with stages, this time we will create 4 stages as follows:</p><pre>stages:<br>- build<br>- review<br>- test<br>- deploy</pre><p><strong>Build<br></strong>In this stage, we will build all the microservices images. We will tag them and push them to Docker Registry.</p><p><strong>Review<br></strong>For every commit other than tags, we will create a review/staging environment and deploy it in a staging environment. For example review.result.1.2.3.4.nip.io</p><p><strong>Test<br></strong>Before deploying to production, we need to run all our integration tests here.</p><p><strong>Deploy<br></strong>In this stage, we will deploy our application in a production environment.</p><h3><strong>Deep Dive in .gitlab-ci.yml CI/CD Jobs</strong></h3><p>I have created a few bash helper script functions and declared them in the before_script section of the whole pipeline. Which means they will be available in all jobs before running the script section.</p><p>In our CI/CD Pipeline the first Job will be the build stage, let&#39;s look into its code:</p><pre>k8s-build-production:</pre><pre>  variables:<br>    DOCKER_HOST: tcp://localhost:2375</pre><pre>  stage: build</pre><pre>  script:<br>    - registry_login<br>    - install_compose<br>    - sed_files compose<br>    - docker-compose build --pull --compress --force-rm<br>    - push_registry</pre><pre>  only:<br>    - master</pre><blockquote><a href="https://gitlab.com/docker-microservice/ci-cd-example-voting-app-k8s/-/blob/0b94482c6dd60e529ceed07c39ca73f7d7fc2d5b/.gitlab-ci.yml">Here is the full code of .gitlab-yml</a></blockquote><p>In the script section, we first log into our registry using docker login , then we make sure that we have docker-compose available. Once we are all set with login and docker-compose installation then we need to substitute the environment variables in our docker-compose.yml with Gitlab predefined environment variables for which I have created a helper method sed_files we pass compose as a string variable to this function. This method makes use of Linux SED utility to substitute the variables (you may also use <a href="https://github.com/a8m/envsubst">envsubst</a>).</p><p>Once our docker-compose.yml file is updated, we can run the docker-compose build command to prepare all the images. push_registry the function simply pushes the built images to our Container Registry with appropriate tags.</p><p>Now the build starts consecutively when we make a change in the master branch, however, if we want to bump a new version tag we need to make sure that all docker images are also bumped with the same tags. That is why I created a k8s-tag-production job.</p><pre>k8s-tag-production:</pre><pre>  variables:<br>    DOCKER_HOST: tcp://localhost:2375</pre><pre>  stage: build</pre><pre>  only:<br>    - tags</pre><pre>  script:<br>    - registry_login<br>    - install_compose<br>    - sed_files compose<br>    - docker-compose pull<br>    - push_registry</pre><p>Now this build runs only when a new version is pushed using:</p><pre>$ git tag v1.1.0 -m &#39;Version 1.1.0&#39;<br>$ git push origin v1.1.0</pre><p>It will also log in to Container Registry, install docker-compose and change environment variables in docker-compose.yml using SED. It will then pull all the latest pushed images and push them back while using the new version tags.</p><p>The next in line is the Review deploy stage, in this stage, we will deploy a staging environment of our app. Let us view the code:</p><pre>deploy_review:</pre><pre>  image:<br>    name: lachlanevenson/k8s-kubectl:latest<br>    entrypoint: [&quot;/bin/sh&quot;, &quot;-c&quot;]</pre><pre>  stage: review</pre><pre>  only:<br>    - branches</pre><pre>  except:<br>    - tags</pre><pre>  environment:<br>    name: review/$CI_BUILD_REF_NAME<br>    url: <a href="https://$ci_environment_slug.104.154.109.250.nip.io/">https://$CI_ENVIRONMENT_SLUG.104.154.109.250.nip.io</a><br>    on_stop: stop_review</pre><pre>  script:<br>    - kubectl version<br>    - sed_files deployment<br>    - kubectl apply -f kube-deployment.yml</pre><blockquote><a href="https://gitlab.com/docker-microservice/ci-cd-example-voting-app-k8s/-/blob/0b94482c6dd60e529ceed07c39ca73f7d7fc2d5b/.gitlab-ci.yml">Here is the full code of .gitlab-yml</a></blockquote><p>We need kubectlcli in this stage to deploy our microservices using the manifest file kube-deployment.yml , for this reason, we will use the lachlanevenson/k8s-kubectl:latest image. This stage will only run on branches and not on tags, that is why we used only and except fields. Theenvironment field is the most important for a review stage, as it will create the CI_ENVIRONMENT_SLUG environment variables during the execution of the jobs and utilizing that variable we will configure our ingress resource. The default url of the environment will be routed to our Votes microservices using Ingress resources. The on_stop variable takes in the name of the job we want to execute when we press the stop button from the Gitlab Environment Details page.</p><p>Initially, we begin the script execution by listing the kubectl version . We use the SED again to substitute the environment variables from kube-deployment.yml . Once completed, we simply apply the manifest file using kubectl apply command. This will create all the resources mentioned in the kube-deployment.yml.</p><p>Skipping the details of stop_review job, as it simply uses the kubectl delete command to delete all the resources of the review environment.</p><p>Let&#39;s get into the deploy_live job, this will deploy production built using the following manifest:</p><pre>deploy_live:</pre><pre>  image:<br>    name: lachlanevenson/k8s-kubectl:latest<br>    entrypoint: [&quot;/bin/sh&quot;, &quot;-c&quot;]</pre><pre>  stage: deploy</pre><pre>  environment:<br>    name: live<br>    url: https://$CI_ENVIRONMENT_SLUG.104.154.109.250.nip.io</pre><pre>  only:<br>    - tags</pre><pre>  when: manual</pre><pre>  script:<br>    - kubectl version<br>    - sed_files deployment<br>    - kubectl apply -f kube-deployment.yml</pre><blockquote><a href="https://gitlab.com/docker-microservice/ci-cd-example-voting-app-k8s/-/blob/0b94482c6dd60e529ceed07c39ca73f7d7fc2d5b/.gitlab-ci.yml">Here is the full code of .gitlab-yml</a></blockquote><p>The only difference between this and the review stage is that it runs only for the tags with the environment name as live</p><h3><strong>Demo</strong></h3><p>First of all clone the repository <a href="https://gitlab.com/docker-microservice/ci-cd-example-voting-app-k8s">here</a>, add a Kubernetes cluster to it and then to test the CI/CD in action, commit a small change, the review pipeline should start building.</p><pre>$ git add .<br>$ git commit -a -m &#39;Deploying CI/CD Pipelines for the first time&#39;<br>$ git push origin master</pre><p>It will start your first pipeline, you may see it in the CI/CD tab as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CKeVxS08h43IcHh5kfgryw.png" /><figcaption>Running CICD Build after commit</figcaption></figure><p>Just cross-check if everything went well in the pipeline and that a stop_review job is also created with a manual trigger.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NnZ5TU4R88R1KUipdLQw2Q.png" /><figcaption>Review Pipeline Deployed Properly</figcaption></figure><p>Now we open the environments page and check the rollout status. Go to Operations -&gt; Environments</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9hUeiZzjakS_W648BWavzg.png" /><figcaption>Review environment rollout status OK</figcaption></figure><p>To open the application in the browser, click View Deployment .</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pzYHpII0QbPoqSFCZRksxw.png" /><figcaption>View Deployment</figcaption></figure><p>Hooray!!!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HgfnH-XdP_JvJyA82hxH5w.gif" /><figcaption>Apps in Action</figcaption></figure><p>Now, this is our review App, let’s stop the review environment and deploy the live (production) app.</p><p>To stop the review environment, just click theStop button on the previous screen, it will run the stop_review job in our pipeline. The job is configured to delete all Kubernetes resources.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VFGIvmF97X0af-m86WNplA.png" /><figcaption>Review Environment Deleted.</figcaption></figure><p>To deploy live, just tag the commit with a version and push tag:</p><pre>$ git tag v1.3.0 &#39;Bumped Version 3&#39;<br>$ git push origin v1.3.0</pre><p>It will create a new pipeline and start running the k8s-tag-production job.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*W3cimgUiDTGFzsnr0bbbAA.png" /><figcaption>Live environment Pipeline running</figcaption></figure><p>When done, click the deploy_live job and trigger the deployment manually.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pi4KyaVxTYbFt8lf4KfmIg.png" /><figcaption>Deploy Live Manually</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Rvkp2iPOGSJPfQOBSyV9sg.png" /><figcaption>Live environment deployed</figcaption></figure><p>Now let’s check the rollout status in the environments once again</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*m6UARI0VXMlWXUbb1oksbw.png" /><figcaption>100% Rollout</figcaption></figure><p>Next, test the live environment by clicking the Open Live Environment button.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gbjgvkK1K3DDWokt7UfIuw.png" /><figcaption>Open Live Environment</figcaption></figure><p>Ypaaaa!!!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tpx0JdK_ruwNLBHJfLpR7w.gif" /><figcaption>Application Deployed on Live Environment</figcaption></figure><h3>Conclusion</h3><p>Containers have provided us with dramatically more flexibility for running cloud-native applications on physical and virtual infrastructure. But breaking a monolith into a lot of microservices can bring in its own challenges. This is where the Kubernetes Container Orchestration comes in to play. It is important that every modern start-up has a strong repertoire of K8s orchestration from the beginning, otherwise, microservices may make you feel like a shepherd.</p><blockquote><a href="https://gitlab.com/docker-microservice/ci-cd-example-voting-app-k8s">The complete code of this article resides here…</a></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/727/0*Piks8Tu6xUYpF4DU" /></figure><p><strong>Follow us on </strong><a href="https://twitter.com/joinfaun"><strong>Twitter</strong></a><strong> </strong>🐦<strong> and </strong><a href="https://www.facebook.com/faun.dev/"><strong>Facebook</strong></a><strong> </strong>👥<strong> and </strong><a href="https://instagram.com/fauncommunity/"><strong>Instagram</strong></a><strong> </strong>📷 <strong>and join our </strong><a href="https://www.facebook.com/groups/364904580892967/"><strong>Facebook</strong></a><strong> and </strong><a href="https://www.linkedin.com/company/faundev"><strong>Linkedin</strong></a><strong> Groups </strong>💬<strong>.</strong></p><p><strong>To join our community Slack team chat </strong>🗣️ <strong>read our weekly Faun topics </strong>🗞️,<strong> and connect with the community </strong>📣<strong> click here⬇</strong></p><figure><a href="https://www.faun.dev/join/?utm_source=medium.com%2Ffaun&amp;utm_medium=medium&amp;utm_campaign=faunmediumbanner"><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6P3WpLjGv5v1ucm5dgkucg.png" /></a></figure><h4>If this post was helpful, please click the clap 👏 button below a few times to show your support for the author! ⬇</h4><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=72b59496979c" width="1" height="1" alt=""><hr><p><a href="https://faun.pub/application-deployment-using-gitlab-ci-cd-on-managed-kubernetes-cluster-at-gcp-72b59496979c">Application Deployment using Gitlab CI/CD on Managed Kubernetes Cluster at GCP</a> was originally published in <a href="https://faun.pub/">FAUN.dev() 🐾</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Microservices based Application Deployment using Gitlab CI/CD on Docker Swarm at Digital Ocean]]></title>
            <link>https://faun.pub/microservices-based-application-deployment-using-gitlab-ci-cd-on-docker-swarm-at-digital-ocean-586eefb07294?source=rss-9e818bfa5a82------2</link>
            <guid isPermaLink="false">https://medium.com/p/586eefb07294</guid>
            <category><![CDATA[docker-swarm]]></category>
            <category><![CDATA[gitlab-cicd]]></category>
            <category><![CDATA[container-orchestration]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[microservices]]></category>
            <dc:creator><![CDATA[Arslan Ali]]></dc:creator>
            <pubDate>Mon, 02 Mar 2020 19:41:33 GMT</pubDate>
            <atom:updated>2020-03-24T16:37:57.539Z</atom:updated>
            <content:encoded><![CDATA[<h3>Microservices based Application Deployment using Gitlab CI/CD on Docker Swarm</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*5oxQ_Gbc-qQqwXwh" /><figcaption>Photo by <a href="https://unsplash.com/@grohsfabian?utm_source=medium&amp;utm_medium=referral">Fabian Grohs</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>Agenda</h3><p>Using Docker’s official sample <a href="https://github.com/dockersamples/example-voting-app.git">voting app microservices example</a> to implement Gitlab’s CI/CD and deploy them on Docker Swarm will be the agenda. In my next article, I will use the same but deploy it on Kubernetes Cluster. Let’s do this!…</p><ol><li>Create a Repository in a Repository Group</li><li>Create .gitlab-ci.yml for the project.</li><li>Change docker-compose to add image names</li><li>Create a Gitlab-Runner Machine on DigitalOcean using CLI.</li><li>Create a Production Machine on DigitalOcean using CLI</li><li>Create CI/CD Secret Variables</li></ol><h3>Create Repository</h3><p>To create repositories on Gitlab lets first make Repository Group, so press the plus icon on the toolbar and select <em>New Group. </em>Name it as you like, but I am naming it ‘Docker Microservices’. Creating a Repository Group will allow you to create micro repositories for microservices in the future and will help you create secrets or other CI variables useable for the whole group.</p><p>Now let’s create a repository, press the same plus icon on the toolbar and this time select repository. This time select <em>New Repository</em> and then select <em>CI/CD for external repo</em> from the tabs (you may select blank if you want to create your own clone). Copy the git url from <a href="https://github.com/dockersamples/example-voting-app.git">here</a> paste in the <em>Git repository URL</em>, give a name to your project, add some description and then press <em>Create</em>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Y31veG5PIwul2DuO1BxZ8A.png" /><figcaption>Create a repository for CI/CD for external repo</figcaption></figure><p>Once done creating the repository, you will come to the following screen where you can simply press <em>Setup CI/CD</em>. It will create the .gitlab-ci.yml for you and open it in an editor.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8DBBZm0e455pI7EpSfz8kQ.png" /><figcaption>Project Repository created from Git URL</figcaption></figure><h3>Create .gitlab-ci.yml</h3><p>Gitlab CI/CD engine looks for a .gitlab-ci.yml file in order to run the CI/CD pipelines. I found gitlab-ci to be the simplest yet advanced way of building the pipelines for linting, building, testing and even deploying the code. Lets go through a few basic concepts while building the .gitlab-ci for our project.</p><p><strong>Step 1: Choosing the Image<br></strong>This is where you will mention the preferred Docker Image which will be used to run a container to build a particular task, for example, we need a docker:19.03.1 image to run all our CI/CD stages. If you want to run different images for each stage then use image inside the stage definition.</p><pre>image: docker:19.03.1</pre><p><strong>Step 2: Stages<br></strong>An array of CI/CD pipeline stage names, you can mention any names. For example, connect-test, build, build-test, stage, production, deploy, etc.</p><pre>stages:<br>- build<br>- test<br>- deploy</pre><p><strong>Step 3: Variables</strong><br>Define secret variables that you want to use in all stages.</p><pre>## REMOTE HOST DNS or Private IP of the Deployment VM, please change the example.com with your host<br>variables:<br>    REMOTE_HOST: $DEPLOYMENT_HOST_IP</pre><p><strong>Step 4: Stage Definition<br></strong>This is where you will inform the gitlab-runner to perform some task (e.g run scripts to build docker images). Following is the stage where we requested the gitlab-runner to build the docker.</p><pre>docker-build-production:</pre><pre>## Login to Gitlab Provided Container Registry for this project.<br>  ## Install Docker Compose<br>  before_script:<br>  - docker login -u gitlab-ci-token -p &quot;$CI_BUILD_TOKEN&quot; &quot;$CI_REGISTRY&quot;<br>  - apk add --no-cache docker-compose</pre><pre>stage: build</pre><pre>script:<br>  - docker-compose build --pull --compress --force-rm<br>  - docker push &quot;$CI_REGISTRY_IMAGE&quot;/vote:latest<br>  - docker push &quot;$CI_REGISTRY_IMAGE&quot;/result:latest<br>  - docker push &quot;$CI_REGISTRY_IMAGE&quot;/worker:latest</pre><pre>only:<br>  - master</pre><p>The docker-build-production will be the name of the task, to be performed in stage: build. Before running the build script it is required to make sure that we are logged in to the right Container Registry, so I logged in to Gitlab’s provided Project Registry using the predefined variable $CI_BUILD_TOKEN. Since I want to use the docker-compose to build all images so let’s install them on the current container which is building the task (remember each stage runs in a separate container). Next, use the docker-compose build command to build all images. —- pull will pull the previously build images to load the caches and will speed up the build process for each image. The only:master makes sure that this stage will only be triggered once a commit is made only on master branch.</p><blockquote>The complete code of <strong>.gitlab-ci.yml</strong> can be found <a href="https://gitlab.com/docker-microservice/ci-cd-for-example-voting-app/-/blob/master/.gitlab-ci.yml">here</a>.</blockquote><p>As soon as you will save the file and commit the changes, the build should start but we need to make a few more changes in the docker-compose.yml and also create some CI/CD secret variables to make it work.</p><blockquote>One important question where does the build running, that is which runner is running it? <br>So the answer is that Gitlab by default assigns one of its shared runners to every project. But it is highly recommended that you in install your own Gitlab Runner on either your development or any cloud VM, <a href="https://docs.gitlab.com/runner/install/">installation instructions here.</a></blockquote><h3>Make Changes in docker-compose.yml</h3><p>We want to issue a single command to build all microservices using docker-compose build. It creates tags of images in the project_folder_service_name format. This means that it will create the tag for the vote service in the docker-compose.yml as <strong>ci-cd-for-example-voting-app_vote</strong> format, which is not the right format for our case. We want to push the images to our Gitlab’s Container Registry provided for the project. In order to do that we need to add an image variable in our docker-compose.yml file for each service like so.</p><pre>vote:<br>  image: registry.gitlab.com/docker-microservice/ci-cd-for-example-voting-app/vote:latest</pre><p>Docker-Compose will create the image tag for each service image the same as the value of the image variable. In our case, for example, it will create <strong>registry.gitlab.com/docker-microservice/ci-cd-for-example-voting-app/vote:latest</strong> for vote service. You should change the docker-microservice with your Repository Group and ci-cd-for-example-voting-app with your project name.</p><blockquote>Repeat the same for docker-stack.yml as well. The complete code of the docker-compose.yml can be found <a href="https://gitlab.com/docker-microservice/ci-cd-for-example-voting-app/-/blob/master/docker-compose.yml">here.</a></blockquote><h3>Create a Gitlab-Runner Machine on DigitalOcean using docker-machine CLI.</h3><p>Gitlab requires a runner to run all the CI/CD tasks, by default all projects have a pool of Gitlab provided shared runners. The runners’ configuration can be found in the project’s Settings -&gt; CI/CD -&gt; Runners. Let’s create our own runner, you can install the gitlab-runner service on your development machine also. However, let’s install it in a Digital Oceans’ Droplet. We will need <a href="https://docs.docker.com/machine/install-machine/">docker-machine</a> to create a droplet, as it will install the latest stable release of docker and docker-compose and create the security certificates automatically.</p><blockquote>You will need a Digital Ocean account to perform the next few tasks. Click <a href="https://m.do.co/c/972f6ca25894">this link</a> to create an account and get <strong>$100 fee credit</strong>.</blockquote><p>Get the API key for your DO Account by clicking API and then Generate API Token. Save the token in $DOTOKEN environment variable of your development machine. Now run the following command to witness the magic.</p><pre>$ docker-machine create --driver digitalocean --digitalocean-access-token $DOTOKEN runner-vm</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*720zEuGW7n2DgZvEJWxGcw.png" /><figcaption>docker-machine create command output</figcaption></figure><pre>$ docker-machine ls</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*h4_GaV9mx2eUrdNjv0QaqQ.png" /></figure><p>Now let’s take out the env for our runner machine to use the important information in our next steps.</p><pre>$ docker-machine env runner-vm</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Szi1SSSneOVoay-is1SyiA.png" /></figure><p>Now lets get into the runner-vm using ssh.</p><pre>$ docker-machine ssh runner-vm</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fBS2GQPAkYOemVn0FBKJrQ.png" /></figure><p>You can notice from the output that its a Debian Ubuntu Machine with 18.04.4LTS installed. Let’s install the gitlab-runner now.</p><pre>$ sudo curl -L --output /usr/local/bin/gitlab-runner <a href="https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-linux-amd64">https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-linux-amd64</a></pre><pre>$ sudo chmod +x /usr/local/bin/gitlab-runner</pre><pre>$ sudo useradd --comment &#39;GitLab Runner&#39; --create-home gitlab-runner --shell /bin/bash</pre><pre>$ sudo gitlab-runner install --user=gitlab-runner --working-directory=/home/gitlab-runner</pre><pre>$ sudo gitlab-runner start</pre><p>Once all done check status.</p><pre>$ sudo gitlab-runner status</pre><p>The runner is now properly installed, let us register it with our Gitlab account. Go to Groups -&gt; Docker Microservices -&gt; Settings -&gt; CI/CD -&gt; Runners. In the <strong>Setup a Group Runner Manually</strong> section you will find the URL and Token for registering the runner.</p><p>Next, run the following command on the runner to register it:</p><pre>$ sudo gitlab-runner register -n \<br>--url <a href="https://gitlab.com/">https://gitlab.com/</a> \<br>--registration-token REGISTRATION_TOKEN \<br>--executor docker \<br>--description &quot;My Gitlab Runner&quot;</pre><p>Refresh the Gitlab CICD settings page and you will see the runner appear in Available Group Runners.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gtAHNv-N2gl0M5zG3wEIQA.jpeg" /><figcaption>Gitlab Available Group Runner — I have removed the Registration Token</figcaption></figure><p>Since we have the runner available, let’s use it in our project. Go to the Your Project -&gt; Settings -&gt; CI/CD -&gt; Runners. Disable Shared Runners and you will notice at the end of the page a Group Runner is activated for the project already.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5T8ji4whWdrUbYJyUMbVsQ.png" /><figcaption>Group Runners Available for the Project</figcaption></figure><p>Now our Runner Machine is up and running, so let’s set up a deployment machine.</p><h3>Create a Production Machine on DigitalOcean using docker-machine CLI</h3><p>We will use the same docker-machine CLI to create the machine at DO.</p><pre>$ docker-machine create --driver digitalocean --digitalocean-access-token $DOTOKEN deployment-vm</pre><p>Once done, get the config.</p><pre>$ docker-machine config deployment-vm</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gaIERqGCBWLmmj-feRItpQ.png" /></figure><p>Now let’s enable Docker-Swarm mode.</p><pre>$ docker-machine ssh deployment-vm<br>$ docker swarm init --advertise-addr 167.71.95.86<br>$ exit</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JnWhOaBhy67sLwhug5hCgg.png" /></figure><blockquote>Follow the instructions in the above command output if you want to join more swam nodes.</blockquote><h3>Create CI/CD Secret Variables</h3><p>Go to Gitlab and open Your Project -&gt; Settings -&gt; CI/CD -&gt; Variables.</p><p>Now let’s create TLS certificate variables, start with TLSCACERT.</p><pre>pbcopy &lt; /Users/eapple/.docker/machine/machines/deployment-vm/ca.pem</pre><p>Now create TLSCERT</p><pre>pbcopy &lt; /Users/eapple/.docker/machine/machines/deployment-vm/cert.pem</pre><p>and then TLSKEY</p><pre>pbcopy &lt; /Users/eapple/.docker/machine/machines/deployment-vm/key.pem</pre><p>Create the DEPLOYMENT_HOST_IP variable to use in our .gitlab-ci.yml, you can do this by copying the output of below and repeating the above step.</p><pre>$ docker-machine ip deployment-vm</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nA8BVH1w8cFJQwZGIdJjww.png" /><figcaption>ALL Four Variables</figcaption></figure><p>That’s it! Now try and do some changes in your project’s master branch and commit changes. To view, the pipelines go to Gitlab CI/CD of your project.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LuwmxZ8BLG6rR-eU-SKPcw.png" /><figcaption>Deployment Succeeded</figcaption></figure><p>Navigate your browser to the IP of your deployment_vm. You can get it using docker-machine</p><pre>$ docker-machine ip deployment-vm<br>167.71.95.86</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*434p6xl7LsPhznVGtcIk6g.gif" /><figcaption>Docker Swarm in action, notice the Container ID</figcaption></figure><p>Let’s make some changes in our code and commit to master to trigger CI/CD. I will change Cats to Goats and Dogs to Tigers in both Vote and Result microservices.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*M1el6cV3PpJj6mDG-EjUYw.gif" /><figcaption>After Changes and Commit to Master</figcaption></figure><p>Here are the pipeline details, notice the complete build, test, and deploy is taking around 5 minutes. Anything less than 10minutes is fine, if it takes more than 10minutes, try breaking each service into a different micro repository and create its own CI/CD pipeline.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6R6XLvJY93i9DqFVuLkdZg.png" /><figcaption>Pipeline Detail</figcaption></figure><h3>Conclusion</h3><p>Containerization and Container Orchestration is one of the most important design requirements of modern application development. I tried to cover most of the aspects of a single repository microservices CI/CD along with VM Provisioning on the cloud. I believe most of the startups should focus on product development rather than integration and delivery. This post will help even a developer (doesn&#39;t need to be a DevOps) in creating a CI/CD.</p><p>If you read till here and if you like this then please share your kindness with some claps, if you require clarifications on any step please comment below. I can be contacted on arslan.ansari@gmail.com for any advanced feature requirements.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/727/0*Piks8Tu6xUYpF4DU" /></figure><p><strong>Follow us on </strong><a href="https://twitter.com/joinfaun"><strong>Twitter</strong></a><strong> </strong>🐦<strong> and </strong><a href="https://www.facebook.com/faun.dev/"><strong>Facebook</strong></a><strong> </strong>👥<strong> and </strong><a href="https://instagram.com/fauncommunity/"><strong>Instagram</strong></a><strong> </strong>📷 <strong>and join our </strong><a href="https://www.facebook.com/groups/364904580892967/"><strong>Facebook</strong></a><strong> and </strong><a href="https://www.linkedin.com/company/faundev"><strong>Linkedin</strong></a><strong> Groups </strong>💬<strong>.</strong></p><p><strong>To join our community Slack team chat </strong>🗣️ <strong>read our weekly Faun topics </strong>🗞️,<strong> and connect with the community </strong>📣<strong> click here⬇</strong></p><figure><a href="https://www.faun.dev/join/?utm_source=medium.com%2Ffaun&amp;utm_medium=medium&amp;utm_campaign=faunmediumbanner"><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6P3WpLjGv5v1ucm5dgkucg.png" /></a></figure><h4>If this post was helpful, please click the clap 👏 button below a few times to show your support for the author! ⬇</h4><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=586eefb07294" width="1" height="1" alt=""><hr><p><a href="https://faun.pub/microservices-based-application-deployment-using-gitlab-ci-cd-on-docker-swarm-at-digital-ocean-586eefb07294">Microservices based Application Deployment using Gitlab CI/CD on Docker Swarm at Digital Ocean</a> was originally published in <a href="https://faun.pub/">FAUN.dev() 🐾</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Technologies I used for developing an Enterprise App]]></title>
            <link>https://arslanwrites.medium.com/technologies-i-used-for-developing-an-enterprise-app-6913cc448b16?source=rss-9e818bfa5a82------2</link>
            <guid isPermaLink="false">https://medium.com/p/6913cc448b16</guid>
            <category><![CDATA[enterprise-application]]></category>
            <category><![CDATA[meteor]]></category>
            <dc:creator><![CDATA[Arslan Ali]]></dc:creator>
            <pubDate>Mon, 10 Jun 2019 10:39:19 GMT</pubDate>
            <atom:updated>2020-03-27T08:41:01.271Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*s_gdjhqCL5Cz4qxKISCbfA.jpeg" /><figcaption>Fixonclick.com — Top Level Microservices Architecture</figcaption></figure><p>I have been working on this project for the last few years, during this journey, I have tried, tested and developed many different modules and used tons of different technologies to deploy a multi-platform application suite. In my quest for development, I lost almost half of my hairs :) however, I enjoyed each and every day of my life working in these current edge technologies. Just for a record I finished my University in 2004 in CS Engineering but did casual software development in C/C++ and had a mere experience of latest Web Technologies until 2014.</p><h4>Exploring .Net</h4><p>I started off in early 2014 with learning advanced features of .Net, learned its Entity and Identity Frameworks, understood OWIN architecture. It took me around 3 months to understand and develop a fully functional prototype. I actually used a fully functional pre-developed base solution for .Net from <a href="https://aspnetzero.com/">AspNetZero</a> which had comprehensive documentation. The final product was very nice however for developing basic small features it required a lot of boilerplate code. (Their recent version is inspiring, especially with their RAD tooling it will take minutes to develop even the most complex features now)</p><p>But later on, while reading more about latest technologies I found that for developing MVP in lesser time I need a technology which should require less code and which gives a better UX at the same time.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*110UOu6B72Ffj545DfXaUQ.png" /><figcaption>ASP.NET Net Core in a Nutshell</figcaption></figure><h4>MEAN Stack</h4><p>I came across MEAN stack (Mongo, Express, AngularJs, and Node) which helped me developing the app within a few weeks with all the features I required in my MVP because I never had to write tons of boilerplate code only for creating proper DB schemas or creating the back-end APIs. By the way, Yomen generators helped me a lot in speeding up the process.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/750/1*DR0kmQ2vKfN9rAOxmLKGKA.png" /><figcaption>This is how each technology in MEAN works</figcaption></figure><h4>MeteorJs</h4><p>MEAN was very nice except that we had a plan to develop our I<strong>n-App Chat</strong> and real-time <strong>Notifications</strong> features which would require developing custom web sockets. Instead of using native web sockets I preferred using PubSub which was available in Meteor out of the box. Firebase was also one of the options but at that time Meteor was more mature and had good community support. Secondly, Meteor had first-class support for MongoDB and I still prefer MongoDB over firebase Realtime Database and even over their new flagship Firestore database. Now there are many reasons for which I will write a different article but here I will mention just one decisive reason which is the MongoDB’s aggregation pipeline and its support for Multi-Join using $lookup aggregation operator.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*38ejOnbuN0M0ZnwL3BWgLA.png" /><figcaption>The amazing Meteor’s 1.8 is available with support for MongoDB v4 and Node v12</figcaption></figure><p>Now since the platform is decided and I practiced app development in all of the major available technologies I had to design a robust application design. Within a few days, I was able to prepare the Alpha and we planned to roll it out. I had no experience of DevOps or even Git so I used Meteor’s Galaxy for hosting our application as it just required a single meteor deploy command to deploy the application.</p><p>Now since the app was in Alpha and we started registering initial set of users so we were hoping to get runtime bugs. Another month or so passed in fixing those bug. I took full advantage of Galaxy and its Continuous Deployment capabilities and was able to maintain the web app for 3000+ users.</p><h4>Ionic</h4><p>This is where I started planning to develop our Mobile App. The best way to roll out a mobile app in minimum possible time was to use Ionic. Ionic had its 2nd version available with Angular 2 and we immediately started a new bare Ionic project, used <em>meteors’ client bundle</em> to connect with the server and get the full real-time database capabilities. It took us another 2 months to learn Angular 2 and Ionic all together and built the production-ready mobile app in Jan 2017.</p><h4>Server-Side Rendering</h4><p>By the time we deployed our Ionic apps, I found that our web app is not crawled by Search Engines properly, as I was using WebSockets to transfer data between client and server. Initially we started using prerendering service of Meteor using M<em>eteor-SEO</em> package, however, even that was not providing the required results.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*RxJ6PdXyG27XFPulDA5L2g.png" /><figcaption>Angular Meteor with Angular Universal</figcaption></figure><h4>Upgrade from Angularjs to Angular 4 Universal</h4><p>I had to implement server-side rendering but it was almost impossible with Angularjs. I had to upgrade our web app from Angularjs to Angular 4 Universal. Instead of using the Upgrade path I have decided to reimplement the whole app in Angular 4. It took us around 2 months to do that and we successfully ported our code and implemented Server-side rendering but during the development, we faced huge challenges with build times of Meteorjs. As our project grew big the build time increased from 5sec to even 50sec for just even a small CSS property change.</p><h4>Introduction of Express and Pre-Rendering</h4><p>This is where we break our Server and client-side app into two individual apps. We built our web app client in Express and implemented server-side rendering. While used Meteor DDP to connect Express with Meteor. Built custom cashing for public pages and pre-rendered the frequently fetched pages at server startup. The server-to-server DDP also helped us in keeping the pre-rendered files updated at all time as on each change in a page record it will pre-render that page again. This prerendering of public pages helped us to reduce TTFB from 1200ms to 590ms. However, we had to improve it even more as it was just static files serving after all.</p><h4>Introduction of Nginx</h4><p>Express was not the best option to serve the static assets so this is where I started learning about Nginx and created a simple nginx.conf file telling it to serve the static asset and fall back to the server if the asset is not available. Implemented the gzip on Nginx but then I decided to keep the gzipped file in the asset as well and removed the compression from Nginx which overall improved the TTFB and brought it down to 180ms for pre-rendered files. We immediately started noticing the growth in our SEO ranking. We have seen 600% overall growth in 3 months in ranks and 800% in impressions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TrNJZqECEj0eVuJDeNKtNQ.png" /></figure><h4>Mobile Web</h4><p>We had our Ionic mobile apps deployed already. All we need was to create a service worker and implement the server-side redirections for our mobile web to work. After just a few tweaks in the code, we implemented the platform-specific logics to handle logins, image uploading, location and push notifications.</p><p>The hybrid mobile app was built for both Service Providers and the Consumers. By the time the users increased the app started getting slow.</p><h4>Mobile Native Performance</h4><p>The hybrid mobile app was built in Ionic which uses WebView to load the UI mobiles. The UI performance on the mobile web was undoubtedly better than a responsive app however the performance was competitively much slower than the native apps. This is where I decided to implement at least the Service Provider part of the app in a Native platform with a single codebase. I had 3 options to explore which included <em>React Native, Native Script</em> and <em>Flutter</em>, I made a todos app in all three of them to get the taste of all of them.</p><h4>Why Flutter</h4><p>It took me literally 3 hours to make a basic todos app from setting and installing the framework to actual coding and creating a production release for both IOS and Android platforms.</p><p>The whole developer experience was out of this world. Dart the new C Styled language was amazing in writing code, toolings and hot reloads. The IDE literally helped build the UI by suggesting the required widgets, it was, in fact, one step ahead of normal intelligence. Flutter itself was amazing, it has comprehensive documentation with a complete widget catalog and has a widget for just about everything. It took me approximately a weeks time to build a complete production grade app in Flutter even though I initially struggled to setup Meteor DDP in Dart. But later I used Bloc, RxDart and even DartRedux to manage my state. Implemented push notifications using Firebase Dart SDK, used image and location plugins wrote my own packages for handling complex forms and published them on pub.dev.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9_xGD2q0-OV-Wr7ArDFf9A.png" /><figcaption>Flutter UI Framework — Beyond Mobile</figcaption></figure><h4>Integration with 3rd Party SaaS</h4><p>No Web or Mobile Application can Live alone, you would always want to integrate it with social media platforms, if you have a proper monetization in place then you might want to integrate it with payment gateways and with bookkeeping services and the list goes on. I will list a few starting with Google Tag Manager, which is one of the most important if you want to implement many different social media and site analytics integrations. Then come the monetization and implementation of payment gateways. With no prior experience of payment gateways and their integration, it was initially appearing to be the most hideous task but once I started reading the API documentation of our gateway it became as easy as writing a file upload api. We used the third party Book-keeping SaaS and also Subscription SaaS instead of reinventing the wheel. However, I have a plan to implement at least the Subscription part in the near future.</p><h4>Integration of Amazon SES and SNS</h4><p>By the time I was writing this article, our AWS SES account got blocked because of increasing Bounce Rate. It is because we never had an Email Bounce and Complaints Management mechanism. I requested Amazon team to reopen my account by telling them my current issue and they were so generous that they opened the account within a few hours. But now I have configured my SES to trigger SNS publication and I have subscribed the same on my notification server to keep a record of all bounced, complaints and other email events. It is worth mentioning that it took just a couple of days to implement a service similar to mail chimp or constant contact management app.</p><h4>Epics in our Product Backlog</h4><ol><li>Flutter App for Customers</li><li>Flutter App for Admins and Stakeholders</li><li>Client-Side Application for Marketing Campaign Management</li><li>Complete Redesign of DevOps infrastructure using JenkinsX and Kubernetes.</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6913cc448b16" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Learning and Creating an Enterprise Application Suite]]></title>
            <link>https://medium.com/series/learning-and-creating-an-enterprise-application-suite-2f98d7d8dbf7?source=rss-9e818bfa5a82------2</link>
            <guid isPermaLink="false">https://medium.com/p/2f98d7d8dbf7</guid>
            <dc:creator><![CDATA[Arslan Ali]]></dc:creator>
            <pubDate>Fri, 31 May 2019 12:45:04 GMT</pubDate>
            <atom:updated>2019-05-31T12:45:04.699Z</atom:updated>
            <content:encoded><![CDATA[<p>I graduated in 2003 with a CS degree. After graduation I got into Computer Networks and System Integration discipline. For 14 years I ran a business of Video Surveillance and Audio Visual System Integration. Just a 3 years ago I sat with an old friend where I decided to follow my passion of Software Development once again. This is where I started off with learning .Net, Javascript JQuery, AngularJs, Angular, Dart, Flutter, Nodejs, Meteor, Nestjs and many more technologies. I spent days and nights to learn code and repeat after 6 months I was able to Deploy a MVP and within few days I was able to deploy a full enterprise application across different platforms.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2f98d7d8dbf7" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>