<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Ibraheem Al Saady on Medium]]></title>
        <description><![CDATA[Stories by Ibraheem Al Saady on Medium]]></description>
        <link>https://medium.com/@ibraheemalsaady?source=rss-d0730eae30c7------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*8cGSKYxr6Z2mFt1BtxQlcQ.jpeg</url>
            <title>Stories by Ibraheem Al Saady on Medium</title>
            <link>https://medium.com/@ibraheemalsaady?source=rss-d0730eae30c7------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 08:36:47 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@ibraheemalsaady/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Terraform Operator: Run your Terraform as native Kubernetes objects]]></title>
            <link>https://medium.com/@ibraheemalsaady/terraform-operator-run-your-terraform-as-native-kubernetes-objects-5838eca9d225?source=rss-d0730eae30c7------2</link>
            <guid isPermaLink="false">https://medium.com/p/5838eca9d225</guid>
            <category><![CDATA[cloud-native]]></category>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[terraform]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[cloud]]></category>
            <dc:creator><![CDATA[Ibraheem Al Saady]]></dc:creator>
            <pubDate>Tue, 18 Jan 2022 12:48:41 GMT</pubDate>
            <atom:updated>2022-05-31T07:07:11.256Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TvyFhhaH70wY7S_g-Sphbg.png" /><figcaption>Terraform In Kubernetes</figcaption></figure><p>I’m excited to announce a project I have been working on. The <a href="https://github.com/kuptan/terraform-operator">Terraform Operator</a> is a Kubernetes CRD that will make it possible to run your Terraform module in Kubernetes as a native Kubernetes object. That means there is a YAML file involved :)</p><h4><strong>Motivation</strong></h4><p>I’m a big fan of GitOps, I was looking for ways to run Infrastructure in Kubernetes and specifically my Terraform modules and be able to declaratively define them in my Git repo.</p><p>You might say there are other tools like <a href="https://crossplane.io/">CrossPlane</a> to help you with that, but I was looking for Terraform specifically just because I already have all the configuration and modules ready. I also didn’t want to be tightly coupled to CrossPlane because I also want to reuse these modules outside of Kubernetes</p><h4>Features</h4><ol><li>Specify the version and point to your terraform module</li><li>Define variables and variable files</li><li>Target a specific workspace</li><li>Custom backends and providers configuration</li><li>Module source from private git repositories</li><li>Defined outputs are written as secrets</li></ol><p><strong><em>Update: 30th of May, 2022</em></strong></p><p>More features have been introduced such as</p><ul><li>Dependency management</li><li>Use dependency outputs as input variables on another module</li><li>Defaulting to Kubernetes backend if a custom backend was not provided</li></ul><h4>Setup &amp; Installation</h4><p>Let&#39;s install the controller and have a look at how to define a run/workflow</p><p>Install the terraform operator with helm</p><pre>helm repo add kube-champ https://kuptan.github.io/helm-charts<br>helm install terraform-operator kuptan/terraform-operator</pre><p>Now let&#39;s create our first Terraform run/workflow</p><pre>apiVersion: run.terraform-operator.io/v1alpha1<br>kind: Terraform<br>metadata:<br>  name: first-module<br>spec:<br>  terraformVersion: 1.0.2<br><br>  module:<br>    source: IbraheemAlSaady/test/module<br>    # version: &lt;- optional module version<br>  variables:<br>    - key: length<br>      value: &quot;16&quot;<br>  outputs:<br>    - key: length<br>      moduleOutputName: result</pre><p>The above configuration targets terraform version1.0.2.</p><p>The module IbraheemAlSaady/test/module only generates a random string based on a specified length and outputs the generated string in output called result . The source code of the module can be found <a href="https://github.com/IbraheemAlSaady/terraform-module-test">here</a></p><p>The above will pass the length variable with value 16 as defined, once the run is completed, the output will be stored in a Kubernetes secret with a key length</p><p>Now that we have created and run our first module, you can expand the configuration to include things like variable files, terraform workspace, sources from private git repos, custom backends, providers configuration, etc…</p><p>You can the <a href="https://kuptan.github.io/terraform-operator/examples/">examples here</a> on how to use the CRD. You can also take a deep dive into <a href="https://kuptan.github.io/terraform-operator/design/">how the operator works under the hood</a></p><h4>What’s Next?</h4><p>I’m actively developing the Terraform Operator and working on adding more features. You can follow the <a href="https://github.com/orgs/kuptan/projects/1/views/1?layout=board">project roadmap</a> to see the status of these features</p><p>Pull requests from the open-source community are more than welcome</p><p>Please support and Star the project, it’s definitely a motivation kick :D</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5838eca9d225" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Kubernetes multi-cluster monitoring architecture]]></title>
            <link>https://medium.com/@ibraheemalsaady/kubernetes-multi-cluster-monitoring-architecture-50f0315371a1?source=rss-d0730eae30c7------2</link>
            <guid isPermaLink="false">https://medium.com/p/50f0315371a1</guid>
            <category><![CDATA[cloud]]></category>
            <category><![CDATA[thanos]]></category>
            <category><![CDATA[infrastructure]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[monitoring]]></category>
            <dc:creator><![CDATA[Ibraheem Al Saady]]></dc:creator>
            <pubDate>Sat, 01 Jan 2022 17:07:58 GMT</pubDate>
            <atom:updated>2022-01-02T20:26:04.597Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/982/1*PZs7iL-O0kNFIjrFd7VWiA.png" /></figure><p><em>This article describes the architecture of monitoring multiple Kubernetes clusters. It doesn’t go over implementation and code</em></p><p>Prometheus with Grafana is a popular monitoring setup for Kubernetes clusters. While this works great and is sometimes out of the box for a single cluster, what happens when your fleet of clusters is getting larger and larger?</p><p>What we wanted to achieve is the following:</p><ol><li>Centralized place to view the metrics from all the clusters</li><li>Centralized storage for all the metrics from all the clusters</li><li>Centralized alert manager instance</li><li>Consistent and scalable setup for any new cluster we’re adding</li></ol><p>We decided to go with <strong>Thanos</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1012/1*pEMspEDKKP4ja4K5CmEMPg.png" /></figure><p>In simple words<strong>, Thanos </strong>is a tool that aggregates metrics from multiple Prometheus instances.</p><h4>Let’s talk architecture</h4><p>Here is a diagram of the current architecture, I’ll go through the components and the responsibility of each component</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VI_G5UL757AMAWlWpDJl_Q.png" /></figure><h4><strong>Prometheus Instances</strong></h4><p>Each Prometheus instance will be configured with <strong>external-labels</strong> to identify the instance (e.g: cluster name). <strong>external-labels </strong>are labels that will be added to<strong> </strong>any time series or alerts when communicating with external systems</p><pre>external_labels:<br>  cluster: cluster-a<br>  another_label: somevalue</pre><h4><strong>Thanos Sidecar</strong></h4><p>The sidecar is a container that runs next to the Prometheus instance. It will read the metrics from the Prometheus instance. It will also backup these metrics into the storage backend. Hence the arrows from the sidecars to the storage.</p><p>Here are the <a href="https://thanos.io/tip/thanos/storage.md/">supported storage clients</a> for Thanos</p><h4><strong>Thanos Query</strong></h4><blockquote>The Query component is stateless and horizontally scalable and can be deployed with any number of replicas. Once connected to the <strong>Sidecars</strong>, it automatically detects which Prometheus servers need to be contacted for a given PromQL query.</blockquote><blockquote>Query also implements Prometheus’s official HTTP API and can thus be used with external tools such as Grafana. It also serves a derivative of Prometheus’s UI for ad-hoc querying and stores status</blockquote><p>Now if you notice in the architectural diagram, each cluster has a Thanos query component that points to the local Thanos sidecar.</p><p>Eventually, the Thanos query instance in the observability cluster will point to the Thanos query instances of the other clusters</p><p>Using the <a href="https://github.com/bitnami/charts/tree/master/bitnami/thanos">Thanos helm chart</a>, here is how our Thanos in the observability cluster looks like</p><pre>query:<br>  stores:<br>    - cluster-a.thanos-query.domain.internal:10901<br>    - cluster-b.thanos-query.domain.internal:10901<br>    - thanos-sidecar.monitoring:10901 # &lt;- local thanos sidecar</pre><p><em>The observability query instance will be used in Grafana as the Prometheus source. We will then be able to see the metrics from all the clusters</em></p><h4><strong>Thanos Storegateway</strong></h4><blockquote>As the sidecar backs up data into the object storage of your choice, you can decrease Prometheus retention and store less locally. However we need a way to query all that historical data again. The store gateway does just that by implementing the same gRPC data API as the sidecars but backing it with data it can find in your object storage bucket. Just like sidecars and query nodes, the store gateway exposes StoreAPI and needs to be discovered by Thanos Querier</blockquote><p>We can add the Storegateway to our centralized Thanos query instance</p><pre>query:<br>  stores:<br>    - ... # &lt;- the above configuration<br>    - thanos-storegateway.monitoring:10901</pre><h4>Thanos Compactor</h4><blockquote>A local Prometheus installation periodically compacts older data to improve query efficiency. Since the sidecar backs up data as soon as possible, we need a way to apply the same process to data in the object storage.</blockquote><blockquote>The compactor component simple scans the object storage and processes compaction where required. At the same time it is responsible for creating downsampled copies of data to speed up queries.</blockquote><h3>Alerting</h3><p>Based on our architectural diagram, we will go with a centralized alert manager and leverage the Thanos ruler component</p><h4>Thanos Ruler</h4><blockquote>It does rule and alert evaluation on top of a given Thanos Querier endpoint.</blockquote><p>According to our architectural diagram, we will set up a Thanos ruler in every cluster. That means the ruler will only evaluate the metrics from the query instance in the same cluster.</p><p>Using the <a href="https://github.com/bitnami/charts/tree/master/bitnami/thanos">Thanos helm chart</a>, here is a snippet of our ruler instance configuration</p><pre>ruler:<br>  enabled: true<br>  alertmanagers:<br>    - http://alertmanager.domain.internal<br>  extraFlags<br>    - --label=cluster=&quot;cluster-a&quot;</pre><p>The --label is a label to be added to the alerts/metrics to identify the ruler instance. This is similar to Prometheus external-labels. We will use the cluster label in our alert manager configuration later</p><p>The same setup will be for each Kubernetes cluster (of course with a different cluster name value in the label).</p><p>Now that we’re set, we need to configure our alert manager on how to route the alerts</p><p>Since this is up to you to configure, I will only link to the <a href="https://github.com/prometheus/alertmanager/blob/main/doc/examples/simple.yml">example</a> on GitHub by the Prometheus team</p><h4>Closing Notes</h4><p>I’ve been using Thanos in production for over a year, and it ticked all the points/goals I wanted to achieve.</p><p>Thanos also supports <a href="https://thanos.io/tip/operating/multi-tenancy.md/">multi-tenancy</a> but I haven’t tried that setup yet</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=50f0315371a1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Achieving logging for a multi tenant Kubernetes cluster]]></title>
            <link>https://medium.com/@ibraheemalsaady/achieving-logging-for-a-multi-tenant-kubernetes-cluster-70fb3bfc0069?source=rss-d0730eae30c7------2</link>
            <guid isPermaLink="false">https://medium.com/p/70fb3bfc0069</guid>
            <category><![CDATA[cloud-architecture]]></category>
            <category><![CDATA[logging]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[observability]]></category>
            <dc:creator><![CDATA[Ibraheem Al Saady]]></dc:creator>
            <pubDate>Thu, 12 Nov 2020 13:57:30 GMT</pubDate>
            <atom:updated>2021-12-24T12:57:42.330Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sJGyeXXAoS41ZD4hCp7OzQ.jpeg" /></figure><p>I will not bore you with my history and journey with Kubernetes, so I’ll jump right into the subject.</p><p>This article is not about implementation, but rather the design and the architecture of building a logging system for a multi-tenant cluster</p><h4>Multi-Tenancy Setup</h4><p>I have a Kubernetes cluster where the cluster is shared with tenants. Each tenant is basically a separate namespace secured by RBAC and network policies.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hueqsZ2safvldk9JO5Yh8Q.jpeg" /></figure><p>For best practices on setting up multi-tenant clusters, check the <a href="https://cloud.google.com/kubernetes-engine/docs/concepts/multitenancy-overview">google guide</a>.</p><h4>The Stack</h4><p>Below is the stack I’ve used to achieve the goal</p><p><strong>Loki: </strong>Loki is a horizontally-scalable, highly-available, multi-tenant log aggregation system inspired by Prometheus</p><p><strong>Viewing Logs: </strong>Grafana<strong> </strong>for displaying the logs</p><p><strong>Data Collection:</strong> I found that <a href="https://github.com/banzaicloud/logging-operator">Logging Operator</a> was the best option for a multi-tenancy setup, and it works great with Loki. It also supports many other systems like ElasticSearch, S3, Kafka, and more.</p><h4>How It Works?</h4><p>Logging Operator has few Kubernetes objects that you need to be familiar with before we talk about the setup</p><ul><li><a href="https://cloud.google.com/kubernetes-engine/docs/concepts/multitenancy-overview">Logging Object</a></li><li><a href="https://banzaicloud.com/docs/one-eye/logging-operator/crds/#flows-clusterflows">Flow &amp; ClusterFlow</a></li><li><a href="https://banzaicloud.com/docs/one-eye/logging-operator/crds/#outputs-clusteroutputs">Output &amp; ClusterOutput</a></li></ul><p>Here is an architecture overview of the Logging Operator</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gj1tIvs0_x1n7VCJLAYXQg.png" /></figure><p>Each tenant represents an organization in Grafana, that means for each namespace created in the cluster, and organization needs to be created in Grafana.</p><p>Loki also needs to be deployed with auth_enabled: true flag. That ensures that the logs are separate for each tenant. Loki will require the HTTP header X-Scope-OrgID to be provided in order to be authenticated</p><p>Each namespace will be created with a Flow and an Output object. Flow will pick up logs based on pod labels and Output will push it to Loki</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fCRlMKVxtUIVPQsyvWGu3g.png" /></figure><p>Here is an example of a configuration I have for a logging flow in a namespace</p><pre>apiVersion: logging.banzaicloud.io/v1beta1</pre><pre>kind: Output</pre><pre>metadata:</pre><pre>  name: loki</pre><pre>  namespace: flux-system</pre><pre>spec:</pre><pre>  loki:</pre><pre>    url: http://loki.logging:3100</pre><pre>    tenant: &quot;2&quot;</pre><pre>    configure_kubernetes_labels: true</pre><pre>---</pre><pre>apiVersion: logging.banzaicloud.io/v1beta1</pre><pre>kind: Flow</pre><pre>metadata:</pre><pre>  name: flux-flow</pre><pre>  namespace: flux-system</pre><pre>spec:</pre><pre>  localOutputRefs:</pre><pre>    - loki</pre><pre>  filters:</pre><pre>    - parser:</pre><pre>        remove_key_name_field: true</pre><pre>        reserve_data: true</pre><pre>        parse:</pre><pre>          type: nginx</pre><pre>    - tag_normaliser:</pre><pre>        format: ${pod_name}.${container_name}</pre><pre>  match:</pre><pre>    - select:</pre><pre>        labels:</pre><pre>          app: flux</pre><pre>    - select:</pre><pre>        labels:</pre><pre>          app: helm-operator</pre><p>As you can see, my flow is picking up logs from two different pods, one pod with a label app: flux and the other pod is with a label app: helm-operator</p><p>The logs are pushed to a Loki output with a tenant tenant: &quot;2&quot; . Which is the organization Id in Grafana.</p><h4>Cluster Logs</h4><p>Now that we have talked about the logs per tenant, what about the logs of the whole cluster?</p><p>You can easily do that by using ClusterFlow and ClusterOutput. I would suggest not sending the Logs to Loki, but rather a different output like Elasticsearch for example. You can use these logs for later analysis</p><h4>Update December 2021</h4><p>After some months of using the Logging Operator in production, I’ve encountered some issues with FluentD pod not being able to send the logs due to the FluentBit connection refused error.</p><p>The cause of this issue is not clear to me yet, but my guess is that FluentD is not able to handle the number of logs being sent to it from the FluentBit pods probably because of the memory buffer.</p><p>This however is only regarding the whole cluster flow (all cluster logs) and not tenant flow, but I thought there might be a good place to share this.</p><p>I’ve written a small Gist to show the configuration I have now and it seems to fix my issue. Gist can be found <a href="https://gist.github.com/IbraheemAlSaady/3e4adb1f3192ebf85c9e7906a4a4d615">here</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=70fb3bfc0069" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Sharing a Single Cluster on Google Cloud Kubernetes Engine]]></title>
            <link>https://medium.com/@ibraheemalsaady/sharing-a-single-cluster-on-google-cloud-kubernetes-engine-def1c21ef121?source=rss-d0730eae30c7------2</link>
            <guid isPermaLink="false">https://medium.com/p/def1c21ef121</guid>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[kubernetes-namespaces]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[google-cloud]]></category>
            <category><![CDATA[kubernetes-cluster]]></category>
            <dc:creator><![CDATA[Ibraheem Al Saady]]></dc:creator>
            <pubDate>Sun, 29 Jul 2018 17:43:08 GMT</pubDate>
            <atom:updated>2018-07-30T06:49:09.420Z</atom:updated>
            <content:encoded><![CDATA[<p>In this article, I will be talking about sharing a single cluster as well as sharing configuration of a project I’m working on, on Google Cloud Kubernetes Engine separating staging &amp; production environments using namespaces.</p><h4>What are Namespaces</h4><p><em>Kubernetes supports multiple virtual clusters backed by the same physical cluster. These virtual clusters are called namespaces. (</em><a href="https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/"><em>Documentation Here</em></a><em>).</em></p><p>My use-case for using namespaces is to separate environments as I mentioned, Another use-case for namespaces is in environments with many users spread across multiple teams, or projects.</p><p>My first configuration was that I used multiple physical clusters for each environment. With that, it can get expensive. So having one physical cluster virtually separated by namespaces saved me some money.</p><p>Here is a great video by Sandeep Dinesh explaining the best practices with namespaces. (Highly recommended to watch)</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FxpnZX3if9Tc%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DxpnZX3if9Tc&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FxpnZX3if9Tc%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/7a10b606684a81c43a1510158d5f14fa/href">https://medium.com/media/7a10b606684a81c43a1510158d5f14fa/href</a></iframe><p>So a cluster by default comes with 3 namespaces, <strong>default</strong>, <strong>kube-system</strong>,<strong> </strong>and <strong>kube-public</strong>.</p><ul><li><strong>default</strong>: The default namespace for objects with no other namespace.</li><li><strong>kube-system</strong>: The namespace for objects created by the Kubernetes system.</li><li><strong>kube-public</strong>: The namespace is created automatically and readable by all users.</li></ul><h4>Creating a Namespace</h4><p>You can create a namespace in two ways:</p><p>Using Kubectl command kubectl create namespace [namespace name here] .</p><p>Or using a YML file, like this:</p><pre>apiVersion: v1<br>kind: Namespace<br>metadata: <br>  name: staging<br>  labels:    <br>    name: staging<br>---<br>apiVersion: v1<br>kind: Namespace<br>metadata: <br>  name: production<br>  labels:    <br>    name: production</pre><p>And do kubectl create -f [path-to-namespace-yml-file] . This will create two namespaces, staging and production.</p><p>I personally prefer the YML file, just to have all my configuration in one place.</p><h4>Managing Resources in Namespaces</h4><p>You want your staging environment to have strict memory/CPU usage for it not to take more than what it should have.</p><p>You can do that by defining Resource Quota and Limit Ranges on the namespace itself, so any pod that will be deployed/added, it will take the CPU and memory configuration from the namespace quota.</p><p>Here is my quota YML file configuration</p><pre># setup the quota for all containers in the staging namespace<br>apiVersion: v1<br>kind: ResourceQuota<br>metadata:  <br>  name: quota-staging  <br>  namespace: staging<br>spec:  <br>  hard:    <br>    requests.cpu: 1500m    <br>    requests.memory: 2500Mi    <br>    limits.cpu: 2000m    <br>    limits.memory: 3000Mi</pre><p>And my Limit Range YML file configuration</p><pre>apiVersion: v1<br>kind: LimitRange<br>metadata:  <br>  name: limit-range-staging  <br>  namespace: staging<br>spec:  <br>  limits:  <br>  - default:      <br>      # default cpu limit for each container in the namespace      <br>      cpu: 500m      <br>      # default memory limit for each container in the namespace     <br>      memory: 400Mi    <br>    defaultRequest:      <br>      # default cpu request for each container in the namespace      <br>      cpu: 400m      <br>      # default memory request for each container in the namespace      <br>      memory: 400Mi    <br>    max:      <br>      # maximum cpu limit that the container could have      <br>      cpu: 700m      <br>      # maximum memory limit that the containe could have      <br>      memory: 500Mi    <br>    min:      <br>      # minimum cpu request that a container can have      <br>      cpu: 100m      <br>      # minimum memory request that a container can have      <br>      memory: 100Mi    <br>    type: Container</pre><p><em>For CPU &amp; Memory units, check the documentation </em><a href="https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/#meaning-of-cpu"><em>here</em></a><em>.</em></p><p>A <strong>limit</strong> is the maximum amount of resources that Kubernetes will allow the container to use.<br>A <strong>request</strong> is what the container should have when its deployed.</p><p>For more information, check the documentation <a href="https://kubernetes.io/docs/tasks/administer-cluster/manage-resources/cpu-default-namespace/">here</a> and <a href="https://kubernetes.io/docs/tasks/administer-cluster/manage-resources/memory-default-namespace/">here</a>.</p><p>Here is a great video also by Sandeep Dinesh explaining the resources in Kubernetes (also recommended to watch).</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FxjpHggHKm78%3Fstart%3D57%26feature%3Doembed%26start%3D57&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DxjpHggHKm78&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FxjpHggHKm78%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/89a89aa2528ebb55ab81cd15ebf015b4/href">https://medium.com/media/89a89aa2528ebb55ab81cd15ebf015b4/href</a></iframe><h4>Accessing Services in Another Namespace</h4><p>Namespaces are hidden from each other, but they’re not fully isolated.</p><p>Services in Kubernetes exposes their endpoints using a common pattern. &lt;service-name&gt;.&lt;namespace-name&gt;.svc.cluster.locl . So you can easily access a service from another namespace following the pattern.</p><p><em>When using only the </em><em>&lt;service-name&gt; , by default, it will resolve to the local namespace.</em></p><h4>Creating Secrets in a Namespace</h4><p>You can create your secrets inside the namespace by including the —- namespace flag in the Kubectl command. Pods in the namespace will only access the secrets in their local namespace. All of this is managed by Kubernetes, so you don’t have to worry about it.</p><p><em>Check the </em><a href="https://kubernetes.io/docs/concepts/configuration/secret/"><em>documentation here</em></a><em> on how to create secrets.</em></p><h4>Deploying to the Namespace</h4><p>Say we have a deployment kind in a YML file that we want to deploy to the staging namespace. You have multiple ways to do that. <br>We can either do kubectl create -f deployment.yml --namespace staging , or we can specify the namespace in the YML file like this</p><pre>apiVersion: extensions/v1beta1<br>kind: Deployment<br>metadata:  <br>  selector:  <br>  labels:    <br>    app: api-production  <br>  name: api-production  <br>  <strong>namespace: production</strong></pre><p>Using the <strong>namespace </strong>flag in the kubectl command can get really annoying.</p><h4>Using a Context</h4><p>You can permanently save the namespace for all subsequent kubectl commands in that context. <br>kubectl config set-context $(kubectl config current-context) --namespace=&lt;namespace-name-here&gt;</p><p>Or you can create a new context for that namespace to use it later. <br>kubectl config set-context [context-name] --cluster=[cluster] --user=[user] --namespace=[namespace-name] .</p><blockquote>You can get the information of the cluster and the user doing kubectl config view</blockquote><p>To use the context, you can type kubectl config use-context [context-name] . Now, when you do any operation, it will do it/fetch it from the namespace specified in the context you have created.</p><h4>Role Based Access Control in Namespaces</h4><p>You can restrict user’s access to namespaces. If you want a team to only have access to the resources in the staging environment whether to get, list, watch, create, etc… or all, you can do that with Role Based Access Control (RBAC).</p><p>Since this is kind of another subject, I will cover it later in a future article, Here is the <a href="https://kubernetes.io/docs/reference/access-authn-authz/rbac/">documentation of RBAC</a>.</p><h4>Wrap Up</h4><p><a href="https://gist.github.com/IbraheemAlSaady/ba47c772cbd6914779d93fcdbe1ac0a9">Here is a Gist</a> of my full configuration of my project, it has all the configuration for both environments (staging &amp; production)</p><p>I have another article that talks about setting up CI/CD pipeline with CircleCI 2 and deploying to Kubernetes engine, <a href="https://medium.com/@isaady92_34293/circleci-2-and-google-cloud-kubernetes-306beee2d64c">check it out</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=def1c21ef121" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Algorithms and Data Structures in Interviews]]></title>
            <link>https://medium.com/@ibraheemalsaady/algorithms-and-data-structures-in-interviews-dc4b9d77a30?source=rss-d0730eae30c7------2</link>
            <guid isPermaLink="false">https://medium.com/p/dc4b9d77a30</guid>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[coding-interviews]]></category>
            <dc:creator><![CDATA[Ibraheem Al Saady]]></dc:creator>
            <pubDate>Wed, 25 Jul 2018 11:16:28 GMT</pubDate>
            <atom:updated>2018-07-25T11:16:28.574Z</atom:updated>
            <content:encoded><![CDATA[<p>I have interviewed in many companies, I’ve been asked a lot of questions, but there is a missing piece in most of the companies I have interviewed in.</p><p>Algorithms and data structures. Problem solving can give you an idea on how the candidate thinks of a problem and how they approach it.</p><p>Most of the questions I got asked are technology related, and what’s latest, things like Angular, React, Node, .NET Core, etc….</p><p>I mean these things are important and nice to have, but I personally think the first part at least should be the problem solving, white board kind of a problem.</p><p>Here is a story of my latest interview in an X company.</p><p>The guy who interviewed me, worked for the big guys, Apple, Microsoft, Amazon, etc… I knew it will not be a technology related kind of an interview.</p><p>The interview was basically two parts, problem solving on a white board, and system design.</p><p>The second part wasn’t hard, but I kind of struggled on the problem solving part. The reason why is that problem solving needs practice, the more problems you solve, the better you get. But actually I haven’t done problem solving for a while, and I wasn’t prepared for that.</p><p>Since that day, I’ve been solving problems (one to two problems a day), rewriting algorithms, just for fun. I believe its something every developer should have.</p><p>I will share with you a YouTube channel and a coding problems website so you can practice and have better problem solving skills, they helped me a lot.</p><p>This YouTube channel called <a href="https://www.youtube.com/channel/UCWSYAntBbdd2SLYUqPIxo0w/featured">Byte By Byte</a> is really helpful. Also you can subscribe for this awesome website <a href="https://www.dailycodingproblem.com/">dailycodingproblem</a> to get a problem everyday by email.</p><h4>Conclusion</h4><p>Problem solving is really important, as well as knowledge in latest technologies.</p><p>Practice solving more problems, as well as rewriting some algorithms like binary search, quick sort, merge sort, trees, heaps, graphs, hash-tables, etc… It will help you in understanding how the algorithm actually works.</p><p>The more you practice, the better you get!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dc4b9d77a30" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[CircleCI 2 and Google Cloud Kubernetes]]></title>
            <link>https://medium.com/@ibraheemalsaady/circleci-2-and-google-cloud-kubernetes-306beee2d64c?source=rss-d0730eae30c7------2</link>
            <guid isPermaLink="false">https://medium.com/p/306beee2d64c</guid>
            <category><![CDATA[google-cloud-platform]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[circleci-2]]></category>
            <dc:creator><![CDATA[Ibraheem Al Saady]]></dc:creator>
            <pubDate>Thu, 05 Jul 2018 11:08:43 GMT</pubDate>
            <atom:updated>2018-07-06T11:39:29.283Z</atom:updated>
            <content:encoded><![CDATA[<p>There are many CI/CD tools out there, my favorite is CircleCI.</p><p>CircleCI had their version 2 for a while, but after <em>August 31, 2018</em>, you will no longer be able to use the old CircleCI setup (Version 1). Its a must that you migrate to CircleCI 2.</p><p><em>When you see CircleCI, I’m referring to CircleCI 2.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HfyrJeBwTC4Y99SVkduuNw.png" /></figure><p>In this article, I’m going to talk about few concepts that CircleCI 2 has that I used, as well as sharing my setup for a NodeJS application that uses MySQL and Redis, as well as being deployed on Google Kubernetes Engine. I will break the configuration down bit by bit.</p><p>If you know about CircleCI 2 and you just want to checkout the configuration, <a href="https://gist.github.com/IbraheemAlSaady/b0c92c06971c1182e8410894031580ca">checkout the full Gist her</a>e.</p><h3>CircleCI Concepts</h3><p>Here are few concepts of CircleCI that I have used in my configuration.</p><p><strong>Jobs</strong></p><blockquote>A CricleCI run is comprised of one or more named jobs.</blockquote><p>A job is basically a set of steps/commands to be executed.</p><p><a href="https://circleci.com/docs/2.0/configuration-reference/#jobs">Documentation Here</a></p><h4><strong>Workflows</strong></h4><blockquote>A workflow is a set of rules for defining a collection of jobs and their run order.</blockquote><p>Here is a video that explains workflows and configuring multiple jobs.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2F3V84yEz6HwA%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D3V84yEz6HwA&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2F3V84yEz6HwA%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/12f3b81b15b0d8bad6f54c807363093d/href">https://medium.com/media/12f3b81b15b0d8bad6f54c807363093d/href</a></iframe><p><a href="https://circleci.com/docs/2.0/workflows/#using-workspaces-to-share-data-among-jobs\">Documentation Here</a></p><h4><strong>Contexts</strong></h4><blockquote>Contexts provide a mechanism for securing and sharing environment variables across projects.</blockquote><p><a href="https://circleci.com/docs/2.0/contexts/">Documentation Here</a></p><h4><strong>Workspaces</strong></h4><blockquote>Workspaces are a feature of Workflows and are used to move data from a job in a Workflow to subsequent jobs.</blockquote><p><a href="https://circleci.com/blog/deep-diving-into-circleci-workspaces/">Documentation Here</a></p><h4><strong>Caching</strong></h4><blockquote>Caching is one of the most effective ways to make jobs faster on CircleCI by reusing the data from expensive fetch operations from previous jobs. After the initial job run, future instances of the job will run faster by not redoing work.</blockquote><p><a href="https://circleci.com/docs/2.0/caching/">Documentation Here</a></p><p>That’s pretty much it. You can of course check the CircleCI 2 docs <a href="https://circleci.com/docs/2.0/">here</a>.</p><h3>Google Cloud Authentication</h3><p>In order to deploy your application from CirlceCI to Google Cloud, you need CircleCI to be authenticated. Fortunately, its pretty simple :)</p><ul><li>Go to your project in Google Cloud Console</li><li>Open the menu, and go to <strong>APIs &amp; Service -&gt; Credentials</strong></li><li>Under Credentials tab, <strong>Create Credentials </strong>and make sure its of type <strong>Service Account Key</strong></li><li>Choose <strong>New Service Account </strong>(I usually name it circleci), and make sure its of type JSON.</li></ul><p>When you’re done, a JSON file will be downloaded.</p><p>Copy the JSON file content (Yes all of it). Go to your <strong>project settings in CircleCI</strong>, under <strong>environment variables </strong>create a new environment variable (I usually name it GOOGLE_AUTH), and give it the value of the content of the JSON file.</p><h3>Ready to build?</h3><p>Again, You can check the full configuration in the Gist <a href="https://gist.github.com/IbraheemAlSaady/b0c92c06971c1182e8410894031580ca">here</a>.</p><p><em>All environment variables below are available in a context in my CircleCI account. Also, CircleCI exports some environment variables that you could use. </em><a href="https://circleci.com/docs/2.0/env-vars/#built-in-environment-variables"><em>Check the Full List of CircleCI Exported Environment Variables</em></a><em>.</em></p><h4>Lets break it down</h4><p>We will have two main sections, jobs and workflows</p><pre>version: 2<br>jobs:<br>  ....<br>workflows:<br>  ....</pre><p>Under jobs, we will specify the jobs and what each job does.</p><pre>version: 2<br>jobs:<br>  build: <br>    # changing the main directory<br>    working_directory: ~/your-project-name<br>    docker:<br>      - image: circleci/node:9.11<br>    steps:<br>      - checkout<br>      - run:<br>          name: list everything in the directory<br>          command: ls -a<br>workflows:<br>  ....</pre><p><em>P.S: If you don’t have workflows defined, then a job named </em><strong><em>build </em></strong><em>has to exist.</em></p><p>build Is the name of the job, under build we have few things going on.<br>working_directory tells CircleCI which directory is the working directory.<br><em>docker tells CircleCI that we’re going to use docker.<br></em>image: circleci/node:9.11 tells CircleCI which docker image to use to build the project.<br>steps are the actual commands that we will write.</p><p>As you can see under steps, there is a command called checkout .<br>This tells CircleCI to change the directory to the working_directory that we have specified earlier, which in this case will be the root directory of your project.</p><p>What we’re going to do next is to do<em> npm install</em> to install our dependencies, build our local docker image, cache <em>node_modules </em>to be used by subsequent jobs and later builds using <strong>caching</strong>, and persist our docker image to the<em> docker-cache </em>directory to be used later for pushing our built image to Google Container Registry<em>.</em></p><pre>steps:<br>  - checkout<br>  - setup_remote_docker<br>  - run:          <br>      name: npm install          <br>      command: npm install      <br>  - run:           <br>      name: building docker image          <br>      command: docker build --rm=false -t asia.gcr.io/${PROJECT}/${DOCKER_IMAGE}:$CIRCLE_SHA1 .</pre><pre>  - run:          <br>      name: create docker-cache directory        <br>      command: mkdir -p docker-cache      <br>  - run:          <br>      name: Save docker image to cache          <br>      command: docker save -o docker-cache/built-image.tar asia.gcr.io/${PROJECT}/${DOCKER_IMAGE}:$CIRCLE_SHA1      <br>  <br>  - persist_to_workspace:<br>      root: .<br>      paths:<br>        - docker-cache</pre><pre>  - save_cache:<br>      key: node-modules-{{ checksum &quot;package.json&quot; }}<br>      paths:<br>        - node_modules</pre><h4>Our test Job</h4><p>The next job that will run after build is our test job. Here, we will do npm test basically.<br>We’re also going to restore the cache to get the node_modules folder.</p><p>In my tests, I need MySQL and Redis in order for the tests to pass. Therefor, I will use multiple docker images for MySQL and Redis.<br>In order to connect to MySQL, we will also need to install a MySQL client.</p><p>After that, we will just run our migrations, seed our database, and run our tests.</p><pre>version: 2<br>jobs:<br>  test:    <br>    working_directory: ~/your-project-name    <br>  docker:      <br>    - image: circleci/node:9.11      <br>    - image: circleci/redis      <br>    - image: mysql:5.7        <br>      environment:          <br>        MYSQL_ALLOW_EMPTY_PASSWORD: yes          <br>        MYSQL_USER: root          <br>        MYSQL_DATABASE: your-database-name    <br>   steps:      <br>     - checkout <br>     - restore_cache:<br>         keys:<br>           - node-modules-{{ checksum &quot;package.json&quot; }}<br>     - run:          <br>         name: installing mysql client from apt          <br>         command: sudo apt install -y mysql-client      <br>     - attach_workspace:          <br>         at: .      <br>     - run:          <br>         name: migrate and seed          <br>         command: npm run db:migrate &amp;&amp; npm run db:seed:run      <br>     - run:          <br>         name: npm test          <br>         command: npm test</pre><h4>deploy job</h4><p>The last job we have is the job that is responsible for deploying our application to Google Kubernetes Engine.</p><pre>version: 2<br>jobs:<br>  deploy-staging:    <br>    working_directory: ~/your-project-name    <br>  docker:      <br>    - image: google/cloud-sdk    <br>  steps:      <br>    - checkout      <br>    - setup_remote_docker      <br>    - attach_workspace:          <br>        at: .        <br>    - run:          <br>        name: Load docker image from cache          <br>        command: docker load --input docker-cache/built-image.tar        <br>    - run:          <br>        name: Auth gCloud SDK          <br>        command: bash ./deploy/gcloud-auth.sh      <br>    - run:           <br>        name: tag docker image with latest          <br>        command: docker tag asia.gcr.io/${PROJECT}/${DOCKER_IMAGE}:$CIRCLE_SHA1 asia.gcr.io/${PROJECT}/${DOCKER_IMAGE}:latest      <br>    - run:          <br>        name: Running staging.sh          <br>        command: bash ./deploy/staging.sh</pre><p>We want to use google cloud SDK, that’s why we included the docker image that has google SDK.</p><p>We’re attaching the workspace at <strong>.</strong> so we can basically get the docker-cache directory that was persisted in the build job.</p><p>After that, we’re loading the docker image that we built earlier using docker load command.</p><p>The gcloud-auth.sh is the script responsible for authenticating CircleCI to use google SDK.<br>We’re going to tag the docker image that we have built with latest. After that we’re running the deployment script.</p><p><em>All scripts are available in the Gist I shared earlier.</em></p><h4>The Actual Workflow</h4><pre>version: 2<br>jobs:<br>  ....<br>workflows:  <br>  version: 2  <br>  build_test_deploy:    <br>    jobs:      <br>      - build:          <br>          context: your-context-name      <br>      - test:                  <br>          requires:            <br>            - build      <br>      - deploy-staging:          <br>          context: your-context-name          <br>          requires:            <br>            - test          <br>          filters:            <br>            branches:              <br>              only: develop</pre><p>Our workflow is called build_test_deploy <br>And it has 3 jobs, build, test, and deploy-staging. These jobs will run after each other as test requires build and deploy-staging requires test.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*d1X2QyZvgVakrfx26hhPDg.png" /></figure><p><em>you have ways to run your workflow, check the </em><a href="https://circleci.com/docs/2.0/workflows/#sequential-job-execution-example"><em>documentation here</em></a><em>.</em></p><p>After test is completed, deploy-staging will run only if we’re on the develop branch as you notice the filter.</p><p>That’s it. One more time, you can check the <a href="https://gist.github.com/IbraheemAlSaady/b0c92c06971c1182e8410894031580ca">full Gist here</a>.</p><p>I hope that was useful, and I appreciate any feedback.</p><p>If you have any questions, please put them in the comment section below or email me at <a href="/service/mailto:ibraheem.al-saady@outlook.com?subject=CircleCI%20Question%20-%20Medium">ibraheem.al-saady@outlook.com</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=306beee2d64c" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>