<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Jose Silva on Medium]]></title>
        <description><![CDATA[Stories by Jose Silva on Medium]]></description>
        <link>https://medium.com/@jaferson123?source=rss-d694c37534bf------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*hvldbiecxzvBYlgJ9CvweQ.jpeg</url>
            <title>Stories by Jose Silva on Medium</title>
            <link>https://medium.com/@jaferson123?source=rss-d694c37534bf------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 11:29:30 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@jaferson123/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Object Storage on your k8s cluster using S3 API and SeaweedFS]]></title>
            <link>https://jaferson123.medium.com/object-storage-on-your-k8s-cluster-using-s3-api-and-seaweedfs-8aba0b34f520?source=rss-d694c37534bf------2</link>
            <guid isPermaLink="false">https://medium.com/p/8aba0b34f520</guid>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[object-storage]]></category>
            <category><![CDATA[aws-s3]]></category>
            <category><![CDATA[cloud-native]]></category>
            <dc:creator><![CDATA[Jose Silva]]></dc:creator>
            <pubDate>Tue, 13 Aug 2024 16:10:29 GMT</pubDate>
            <atom:updated>2024-08-14T14:13:39.563Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*JEyzexuWzvmlnZap" /><figcaption>Photo by <a href="https://unsplash.com/@wwarby?utm_source=medium&amp;utm_medium=referral">William Warby</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Object storage is an architecture designed for managing and storing massive volumes of unstructured data. Unlike traditional file systems that organize data in a hierarchical structure or block storage that divides data into fixed-sized blocks, object storage encapsulates data into objects units, the object is composed of, the data itself, customisable metadata, and an unique identifier enabling scalability and making it ideal for cloud storage services, big data analytics, and media repositories.</p><p>Access to object storage is usually done through HTTP RESTful APIs and easily searchable and retrievable via metadata that can host, creation date, permission, location and any relevant information to the data.</p><p>Due to its robust scalability, and ease access, object storage is a preferred solution for modern data storage needs.</p><p>In this article, I will go through the process of installing an object storage service (SeaweedFS) on a Kubernetes cluster and how to interact with it via the S3 API.</p><h3>SeaweedFS</h3><p>First, I’ll give a quick summary of the SeaweedFS architecture. It has three main services, Master, Volume and Filer (Optional).</p><ul><li><strong>Master:</strong> Works as the hard of SeaweedFS and it’s responsible for assigning file ids, appointing which volume to store the objects in and deciding which node are part of the nodes</li><li><strong>Volume:</strong> It’s responsible for storing objects into larger individual blocks on the disk. It also sends back heartbeats with status information to the lead Master service.</li><li><strong>Filer:</strong> Enables the visualisation of objects into URLs or file system, the S3 API can run together as part of the Filer or separated.</li></ul><p>To learn in more details about the SeaweedFS components and architecture visite the SeaweedFS wiki <a href="https://github.com/seaweedfs/seaweedfs/wiki/Components">page</a> .</p><h4>Deploying SeaweedFS on Kubernetes</h4><p>In this example, we are using <a href="https://kind.sigs.k8s.io/">KinD</a> to provision a Kubernetes cluster on your local machine, and we will create two nodes a control plane and one worker node.</p><pre><br>$ cat &lt;&lt;EOF | kind create cluster - name=${CLUSTER_NAME} - config=-<br>kind: Cluster<br>apiVersion: kind.x-k8s.io/v1alpha4<br>nodes:<br>- role: control-plane<br>- role: worker<br>EOF</pre><p>Once the cluster is created we should be able to get the <em>KUBECONFIG</em> and interact with the cluster using <em>kubectl.</em></p><pre>## get the cluster kubeconfig using the kind CLI<br>$ kind get kubeconfig --name=object-storage &gt; object-storage.config<br><br>## export the kubeconfig env to point to the kind cluster we created <br>$ export KUBECONFIF=$(pwd)/object-storage.config<br><br>## check the nodes on the cluster<br>$ kubectl get nodes<br>NAME                           STATUS   ROLES           AGE   VERSION<br>object-storage-control-plane   Ready    control-plane   22d   v1.27.1<br>object-storage-worker          Ready    &lt;none&gt;          22d   v1.27.1</pre><p>Now that our object storage cluster is created, lets create a namespace where we want to deploy the SeaweedFS services.</p><pre>## create object-storage namespace<br>$ kubectl create namespace object-storage</pre><p>To deploy the SeaweedFS services (Master, Volume and Filer) on our cluster, we will be using the <a href="https://github.com/seaweedfs/seaweedfs/tree/master/k8s/charts/seaweedfs">Helm chart</a> provided in the SeaweedFS <a href="https://github.com/seaweedfs/seaweedfs/tree/master/k8s/charts/seaweedfs">repository</a> on Github, if you are not familiar with Helm charts please head over to <a href="https://helm.sh/docs/">Helm documentation website</a> to learn more.</p><blockquote>We could install the chart by creating a helm repository but for simplicity I’ve cloned the repo and installed the chart from my machine as it’s easier to customize.</blockquote><pre>## clone the repo<br>$ git clone git@github.com:seaweedfs/seaweedfs.git ./<br><br>## install the chart<br>$ helm upgrade --install seaweedfs ./seaweedfs/seaweedfs/k8s/charts/seaweedfs \<br>--values=./seaweedfs/seaweedfs/k8s/charts/seaweedfs/values.yaml \<br>--namespace object-storage<br><br>$ helm ls -n object-storage<br>NAME      NAMESPACE     REVISION UPDATED  STATUS   CHART  APP VERSION<br>seaweedfs object-storage 1        2024-06-17 17:13:39.538025 +0100 IST deployed seaweedfs-3.67.1 3.67</pre><p>The Helm chart will deploy three StatefulSet resources (Master, Volume and Filer) on the object-storage namespace and three pods should be running.</p><pre>$ kubectl get statefulset -n object-storage<br>NAME               READY   AGE<br>seaweedfs-filer    1/1     30s<br>seaweedfs-master   1/1     30s<br>seaweedfs-volume   1/1     30s<br><br>$ kubectl get po -n object-storage<br>NAME                 READY   STATUS    RESTARTS   AGE<br>seaweedfs-filer-0    1/1     Running   0          30s<br>seaweedfs-master-0   1/1     Running   0          30s<br>seaweedfs-volume-0   1/1     Running   0          30s</pre><blockquote>Note: We have changed some of the default configuration in the values.yaml to simplify the deployment of the services:</blockquote><blockquote>- <a href="https://github.com/seaweedfs/seaweedfs/blob/master/k8s/charts/seaweedfs/values.yaml#L232">Disabled all the readiness and liveness probs</a><br>- <a href="https://github.com/seaweedfs/seaweedfs/blob/master/k8s/charts/seaweedfs/values.yaml#L150">Commented out affinity rules</a><br>- <a href="https://github.com/seaweedfs/seaweedfs/blob/master/k8s/charts/seaweedfs/values.yaml#L168">Commented out node selector</a></blockquote><p>For this example we are creating <em>PersistentVolumeClaim</em> (PVC) in kubernetes to store the data and metadata generated by <em>SeaweedFS</em>, so you should be able to find two PVCs created for the Filer and Volume services.</p><pre>$ kubectl get pvc -n object-store<br>NAME                           STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE<br>data-filer-seaweedfs-filer-0   Bound    pvc-a2747c46-77e1-4f67-963c-6e7ce7d63e8b   5Gi        RWO            standard       9d<br>data1-seaweedfs-volume-0       Bound    pvc-1155ad52-e58c-42cb-acad-ed756e4abecf   50Gi       RWO            standard       9d</pre><blockquote>By default KinD uses <a href="https://github.com/rancher/local-path-provisioner">local path provisioner</a> to utilise local storage on the nodes.</blockquote><h4>Interacting with the S3 API</h4><p>Now we can start interacting with our Object Storage service using the S3 API.</p><p>First we need to get the credentials to authenticate with the AWS CLI, the credentials can be found on the <em>seaweedfs-s3-secret</em> Secret resource.</p><pre>$ kubectl get secret -n object-storage seaweedfs-s3-secret -oyaml<br>apiVersion: v1<br>data:<br>  admin_access_key: <br>  admin_access_key_id: <br>  admin_secret_access_key: <br>...<br>kind: Secret<br>metadata:</pre><p>Then we decode the secret and enter the credentials to authenticate with the AWS CLI.</p><pre>## To decode the credentails from the secret you can use the command below<br>$ kubectl get secret -n object-storage seaweedfs-s3-secret -oyaml | yq -r &quot;.data.admin_access_key_id | base64 -d&quot;<br><br>## configure credentials wit AWS CLI<br>$ aws configure<br>AWS Access Key ID [****************eoWh]: <br>AWS Secret Access Key [****************oXi9]:<br>Default region name [test]:<br>Default output format [None]:</pre><p>To make the S3 API accessible we need to port forward the S3 service, to do so open an extra tab on the terminal and run the command below.</p><pre>$ kubectl port-forward -n object-storage svc/seaweedfs-s3 8333<br>Forwarding from 127.0.0.1:8333 -&gt; 8333<br>Forwarding from [::1]:8333 -&gt; 8333</pre><p>Go back to the main terminal tab where we authenticated with the AWS CLI and start using the S3 commands.</p><p>In this example below, we are going to create a bucket and push a file to it.</p><pre>## create a new bucket<br>$ aws --endpoint-url http://localhost:8333 s3 mb s3://jose<br>make_bucket: jose2<br><br>## list the buckets<br>$ aws --endpoint-url http://localhost:8333 s3 ls<br>2024-07-10 15:18:14 jose<br>2024-07-19 17:03:58 jose2<br><br>## write to a file<br>$ echo &quot;blablabla&quot; &gt; text.txt<br><br>## upload a file to a bucket<br>$ aws --endpoint-url http://localhost:8333 s3 cp ./test.txt s3://jose/test.txt<br>upload: ./test.txt to s3://jose/test.txt<br><br>## list the file on the bucket <br>$ aws --endpoint-url http://localhost:8333 s3 ls s3://jose</pre><h3>Conclusion</h3><p>Object storage is an excellent solution for handling large volumes of data on Kubernetes. Its scalability, combined with the speed at which data can be searched and retrieved, makes it an ideal fit for containerized applications.</p><ul><li><a href="https://github.com/seaweedfs/seaweedfs">GitHub - seaweedfs/seaweedfs: SeaweedFS is a distributed storage system for object storage (S3), file systems, and Iceberg tables, designed to handle billions of files with O(1) disk access and effortless horizontal scaling.</a></li><li><a href="https://helm.sh/docs/">Docs Home</a></li><li><a href="https://docs.aws.amazon.com/AmazonS3/latest/API/Welcome.html">Welcome</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8aba0b34f520" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Dealing with race condition in Golang using channels]]></title>
            <link>https://jaferson123.medium.com/dealing-with-race-condition-in-golang-using-channels-456c0ab09e5b?source=rss-d694c37534bf------2</link>
            <guid isPermaLink="false">https://medium.com/p/456c0ab09e5b</guid>
            <category><![CDATA[race-condition]]></category>
            <category><![CDATA[concurrency]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[coding]]></category>
            <dc:creator><![CDATA[Jose Silva]]></dc:creator>
            <pubDate>Wed, 29 May 2024 14:49:24 GMT</pubDate>
            <atom:updated>2024-05-29T14:51:31.424Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="Shanghai interchange, Unsplash License" src="https://cdn-images-1.medium.com/max/1024/1*-_p5z5lewMvVU2xqbn61Cg.jpeg" /><figcaption>Shanghai interchange <a href="https://unsplash.com/photos/aerial-photography-of-concrete-roads-7nrsVjvALnA">Unsplash</a></figcaption></figure><p>Concurrent code is usually one of the most difficult to understand, as it involves multiple tasks or processes running simultaneously and interacting with multiple resources such as memory, files, or network connections. This can lead to countless issues in code such as Deadlocks, Livelocks, Starvation and Race conditions.</p><p>In this blog post I will be explaining race conditions and how we can prevent them in Golang…</p><h3>What is Race Condition?</h3><p>Race condition occur where multiple processes need to execute in a specific order to produce the intended outcome. For instance, one process is trying to read from a variable while another processe is simultaneously writing to it. If the execution of the program does not ensure that these processes run in the expected sequence a race condition can arise.</p><p>Lets translate the problem mentioned about into code…</p><pre>package main<br><br>import &quot;fmt&quot;<br><br>func main() {<br> var greeting = &quot;Hello&quot;<br><br> go func() {<br>  greeting = &quot;Hello from Brazil&quot;<br> }()<br><br> fmt.Println(greeting)<br>}</pre><p>As we can see in the provided code snippet, despite assigning “<em>Hello from Brazil”</em> to the <em>greeting</em> variable, the output will be “<em>Hello”</em>.</p><p>This scenario exemplifies a common issue known as a data race. In a data race, one process attempts to write to a variable while, at an unspecified time, another process attempts to read from it.</p><h3>How to prevent?</h3><p>There are numerous ways to prevent race conditions, including the use of locking mechanisms such as Mutexes, Atomic Operations which ensure synchronous processing, Immutable data structures that maintain consistency by disallowing changes after creation, and various other techniques.</p><p>One of the Go creators, Rob Pike, describes the quote below, to emphasise one of the best feature in the language, <a href="https://go.dev/tour/concurrency/1">Goroutines</a> and how communication should be done between them.</p><blockquote>Do not communicate by sharing memory; instead, share memory by communicating. Rob Pike</blockquote><p>Sharing memory by communicating highlights the significance of utilising channels to facilitate data transfer between Goroutines. This approach avoids the potential for race conditions by preventing direct access to the same memory address, making your concurrent program safer and more reliable.</p><p>Let’s use the same scenario described in the code snippets above but now making usage of channels to communicate between the Goroutines</p><pre>package main<br><br>import &quot;fmt&quot;<br><br>func main() {<br> var greeting = &quot;Hello&quot;<br> greetingChan := make(chan string)<br><br> go func() {<br>  greeting = &quot;Hello from Brazil&quot;<br>  greetingChan &lt;- greeting<br>  close(greetingChan)<br> }()<br><br> fmt.Println(&lt;-greetingChan)<br>}</pre><p>Running the code above results in the output `<em>Hello from Brazil</em>`. This occurs because Channels operate as blocking processes; when a Goroutine reads from a Channel, it blocks until another Goroutine writes to channel and subsequently closes it, thereby unblocking the process.</p><p>If you’re new to concurrency in Golang, I recommend going to the <a href="https://go.dev/tour/concurrency/1">official documentation</a> and delve into how Go manages concurrent code, particularly focusing on channels.</p><h3>Conclusion</h3><p>Understanding concurrency in Go is essential for preventing race condition and by leveraging the features available in the language we can mitigate problems and build resilient and high-performing concurrent applications.</p><p><a href="https://golangprojectstructure.com/rob-pike-go-proverbs/#dont-communicate-by-sharing-memory-share-memory-by-communicating">https://golangprojectstructure.com/rob-pike-go-proverbs/#dont-communicate-by-sharing-memory-share-memory-by-communicating</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=456c0ab09e5b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Rate limiting with MARIN3R Operator]]></title>
            <link>https://itnext.io/rate-limiting-with-marin3r-operator-5112858871be?source=rss-d694c37534bf------2</link>
            <guid isPermaLink="false">https://medium.com/p/5112858871be</guid>
            <category><![CDATA[envoy-proxy]]></category>
            <category><![CDATA[openshift]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[rate-limiting]]></category>
            <category><![CDATA[kubernetes-operator]]></category>
            <dc:creator><![CDATA[Jose Silva]]></dc:creator>
            <pubDate>Mon, 10 May 2021 20:57:20 GMT</pubDate>
            <atom:updated>2021-05-15T13:11:48.810Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*COkyB0OPb8FnxNGkFP8q-w.jpeg" /><figcaption><a href="https://www.dw.com/en/germans-at-loggerheads-over-autobahn-speed-limits/a-47253578">https://www.dw.com/en/germans-at-loggerheads-over-autobahn-speed-limits/a-47253578</a></figcaption></figure><p>Recently I was working on a project where one of the requirements was to include more control over the number of incoming requests to protect services from attacks and prevent resources from being overloaded and eventually crash. The solution for this was to rate limit key services in the system to secure the application from undesired requests.</p><p>In this blog, I will explain what rate limit is responsible for, why we need this in an application, and talk about the approach we used to add a centralized rate limiting service that runs on an OpenShift cluster.</p><h4>What is rate-limiting?</h4><p>Rate limiting is a technique used to control the rate of requests coming and going from a network, for example, if you are using an API that allows a certain number of requests per minute and If this limit is exceeded, any other request that comes within a minute will be denied and a <em>429 — Too Many Request error</em> will be sent back in response to the request.</p><p>It is important for preventing a server from reaching its capacity or protecting against attacks like <a href="https://www.paloaltonetworks.com/cyberpedia/what-is-a-denial-of-service-attack-dos">DDoS</a> or any malicious bot activity. So by implementing rate limiting we can ensure that services will not be overloaded with unwanted requests and will not consume all the resources available on the server.</p><p>We can also define metrics to monitor the number of requests that are currently being processed or blocked and investigate whether there is a need for scaling up resources in the server to enable more requests to be processed. There are several ways of implementing rate limits, it can be done via the application, web server, caching, etc… We will be covering how to implement rate limits by proxying requests and counting them toward the total number of requests permitted.</p><p><strong>Implementing Rate Limit with MARIN3R Operator</strong></p><p>To get a rate limit service working, we first need to have control of every request that comes through the network, for this, we can use <a href="https://www.envoyproxy.io/">Envoy</a> as a proxy server to intercept and respond to incoming requests before they reach the application services.</p><p>We are going to use <a href="https://github.com/3scale/marin3r">MARIN3R Operator</a> to facilitate the deployment of sidecar containers with Envoy running and configured to first receive requests and then forward them onto our application container, it may sound a bit confusing but I will be demonstrating how it works later on. MARIN3R allows us to define which services will have the proxy container injected by adding annotations and labels to the deployment object of our applications and we can dynamically configure those proxies by creating EnvoyConfig CRDs.</p><p>Along with the MARIN3R operator, an envoy <a href="https://github.com/envoyproxy/ratelimit">global rate limit service</a> with a Redis database will be deployed to our cluster, to control and store the total amount of requests processed and denied.</p><p><strong>Summarizing</strong></p><p>These are all the services we will be using to add rate limiting to our application:</p><ul><li>Envoy proxy</li><li>Global rate limit service</li><li>Redis database</li><li>MARIN3R Operator</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/641/0*bYav9ERaMZw8xApu" /><figcaption>Diagram showing how the services will be integrated</figcaption></figure><h4>Demo</h4><p>Before starting progressing with the demo do make sure you have access to the following:</p><ul><li>OpenShift or Kubernetes cluster</li><li>GitHub</li><li><a href="https://docs.openshift.com/container-platform/4.7/cli_reference/openshift_cli/getting-started-cli.html#installing-openshift-cli"><em>oc</em></a> or <a href="https://kubernetes.io/docs/tasks/tools/"><em>kubectl</em></a> CLI</li></ul><p>In this demo, I will be using an OpenShift cluster but the same can be done on a Kubernetes cluster.</p><ul><li>The first thing we are going to do is deploy a small rest application that will be running alongside the envoy proxy container to rate-limit incoming requests.</li></ul><pre>## Create application namespace<br>oc new-project rest-application<br><br>## Deploy the application<br>cat &lt;&lt;&#39;EOF&#39; | kubectl apply -f -<br>---<br>apiVersion: apps/v1<br>kind: Deployment<br>metadata:<br>  name: httpbin<br>  namespace: rest-application<br>  labels:<br>    app: httpbin<br>spec:<br>  selector:<br>    matchLabels:<br>      app: httpbin<br>  template:<br>    metadata:<br>      labels:<br>        app: httpbin<br>    spec:<br>      containers:<br>        - name: httpbin<br>          image: jsmadis/httpbin<br>          ports:<br>            - containerPort: 8080<br>              protocol: TCP<br>---<br>kind: Service<br>apiVersion: v1<br>metadata:<br>  name: httpbin<br>  namespace: rest-application<br>  labels:<br>    app: httpbin<br>spec:<br>  ports:<br>    - name: 8080-tcp<br>      protocol: TCP<br>      port: 8080<br>      targetPort: 8080-tcp<br>  selector:<br>    app: httpbin<br>EOF<br><br>## expose service<br>oc expose svc/httpbin<br><br>## verify if application is running<br>oc get pods<br>NAME               READY   STATUS      RESTARTS   AGE<br>httpbin-1-8tpdh    1/1     Running     0          53s<br>httpbin-1-deploy   0/1     Completed   0          55s</pre><p>The commands above deploy a REST API application on the cluster and you should be able to access the application by getting the host from its route object:</p><pre>$ oc get route httpbin -o json | jq .spec.host</pre><ul><li>Once the application is up and running clone MARIN3R Operator onto a folder in your local machine:</li></ul><pre>$ git clone https://github.com/3scale/marin3r.git</pre><ul><li>Install MARIN3R Operator on the cluster, I will be using <a href="https://github.com/operator-framework/operator-lifecycle-manager">OLM</a> to install the operator, but the same can be done with <a href="https://github.com/3scale-ops/marin3r#install-using-kustomize">Kustomize</a> in case OLM is not installed on your cluster:</li></ul><pre>$ oc apply -f examples/quickstart/olm-install.yaml</pre><p>This will create 4 resources on the cluster Namespace, CatalogSource, OperatorGroup and Subscription to trigger the installation of MARIN3R via OLM.</p><ul><li>Wait until all MARIN3R services are in a running state:</li></ul><pre>$ oc -n marin3r-system get pods | grep Running</pre><ul><li>We will need to patch the operatorgroup resource to add the namespace of our REST API application to the targedNamespace list, this will allow MARIN3R to watch resources in our REST API namespace:</li></ul><pre>$ oc patch operatorgroup marin3r-system -n marin3r-system --type json -p=&#39;[{&quot;op&quot;: &quot;add&quot;, &quot;path&quot;: &quot;/spec/targetNamespaces&quot;, &quot;value&quot;: [&quot;rest-application&quot;]}]&#39;</pre><ul><li>To dynamically inject the Envoy sidecar container, we need to deploy the discovery service on the namespace where the application is running, for our example we will create the discovery service on the rest-application namespace:</li></ul><pre>cat &lt;&lt;&#39;EOF&#39; | kubectl apply -f -<br>apiVersion: operator.marin3r.3scale.net/v1alpha1<br>kind: DiscoveryService<br>metadata:<br>  name: discoveryservice<br>  namespace: rest-application<br>EOF</pre><ul><li>Add MARIN3R labels and annotations to the application DeploymentConfig object, this will activate the injection of the envoy sidecar container to the Pod’s spec:</li></ul><pre>$ oc patch deployment httpbin -n rest-application -p &#39;{&quot;spec&quot;:{&quot;template&quot;:{&quot;metadata&quot;:{&quot;labels&quot;:{&quot;marin3r.3scale.net/status&quot;:&quot;enabled&quot;}, &quot;annotations&quot;: {&quot;marin3r.3scale.net/node-id&quot;: &quot;httpbin&quot;,&quot;marin3r.3scale.net/ports&quot;: &quot;8080-tcp:8443&quot;}}}}}&#39;</pre><p>We should see two containers running in our application’s pod</p><pre>oc get pods -l app=httpbin<br>NAME              READY   STATUS    RESTARTS   AGE<br>httpbin-3-c4mpz   2/2     Running   0          3m31s</pre><ul><li>We need to configure Envoy proxy to redirect any incoming requests to our application, when we added the label marin3r.3scale.net/ports to 8080-tcp:8443 we are saying that any request coming from the service port 8080-tcp will be redirected to port 8443 The envoy configuration will be listening to 8443 port as you can see below:</li></ul><pre>cat &lt;&lt;&#39;EOF&#39; | kubectl apply -f -<br>apiVersion: marin3r.3scale.net/v1alpha1<br>kind: EnvoyConfig<br>metadata:<br>  name: httpbin<br>spec:<br>  nodeID: httpbin<br>  serialization: yaml<br>  envoyResources:<br>    clusters:<br>      - name: httpbin<br>        value: |<br>          name: httpbin<br>          connect_timeout: 2s<br>          type: STRICT_DNS<br>          lb_policy: ROUND_ROBIN<br>          load_assignment:<br>            cluster_name: httpbin<br>            endpoints:<br>              - lb_endpoints:<br>                  - endpoint:<br>                      address:<br>                        socket_address:<br>                          address: 127.0.0.1<br>                          port_value: 8080<br>    listeners:<br>      - name: http<br>        value: |<br>          name: http<br>          address:<br>            socket_address:<br>              address: 0.0.0.0<br>              port_value: 8443<br>          filter_chains:<br>            - filters:<br>              - name: envoy.http_connection_manager<br>                typed_config:<br>                  &quot;@type&quot;: type.googleapis.com/envoy.config.filter.network.http_connection_manager.v2.HttpConnectionManager<br>                  stat_prefix: ingress_http<br>                  route_config:<br>                    name: local_route<br>                    virtual_hosts:<br>                      - name: httpbin<br>                        domains: [&quot;*&quot;]<br>                        routes:<br>                          - match:<br>                              prefix: &quot;/&quot;<br>                            route:<br>                              cluster: httpbin<br>                  http_filters:<br>                    - name: envoy.router<br>EOF</pre><p>If we curl our application endpoint, we will see that an envoy header was added to the response which means that requests are being proxied:</p><pre>$ curl -I &quot;http://$(oc get route httpbin -o json | jq -r .spec.host)&quot;</pre><ul><li>Create a ConfigMap to define how we will be limiting the requests, you can learn more about it <a href="https://github.com/envoyproxy/ratelimit#configuration">here</a>:</li></ul><pre>cat &lt;&lt;&#39;EOF&#39; | oc apply -f -<br>apiVersion: v1<br>kind: ConfigMap<br>metadata:<br>  name: ratelimit-config<br>  namespace: marin3r-system<br>  labels:<br>    app: ratelimit<br>data:<br>  kuard.yaml: |<br>    domain: httpbin<br>    descriptors:<br>      - key: generic_key<br>        value: slowpath<br>        rate_limit:<br>          unit: minute<br>          requests_per_unit: 1<br>EOF</pre><p><em>In this example, we are defining that only one request per minute will be allowed to our application&#39;s routes</em></p><ul><li>Deploy a Redis database, you can find out more on how to configure Redis <a href="https://github.com/envoyproxy/ratelimit#redis">here</a>:</li></ul><pre>cat &lt;&lt;&#39;EOF&#39; | oc apply -f -<br>apiVersion: apps/v1<br>kind: Deployment<br>metadata:<br>  name: redis<br>  namespace: marin3r-system<br>  labels:<br>    app: redis<br>spec:<br>  selector:<br>    matchLabels:<br>      app: redis<br>  template:<br>    metadata:<br>      labels:<br>        app: redis<br>    spec:<br>      containers:<br>        - name: redis<br>          image: redis:alpine<br>          ports:<br>            - containerPort: 6379<br>              name: redis<br>              protocol: TCP<br>---<br>apiVersion: v1<br>kind: Service<br>metadata:<br>  name: redis-service<br>    namespace: marin3r-system<br>  labels:<br>    app: redis<br>spec:<br>  type: ClusterIP<br>  ports:<br>    - name: envoy-https<br>      port: 6379<br>      targetPort: 6379<br>  selector:<br>    app: redis<br>EOF</pre><ul><li>Deploy the rate limit application onto the cluster:</li></ul><pre>cat &lt;&lt;&#39;EOF&#39; | oc apply -f -<br>apiVersion: apps/v1<br>kind: Deployment<br>metadata:<br>  labels:<br>    app: ratelimit<br>  name: ratelimit<br>  namespace: marin3r-system<br>spec:<br>  replicas: 1<br>  selector:<br>    matchLabels:<br>      app: ratelimit<br>  strategy:<br>    type: Recreate<br>  template:<br>    metadata:<br>      labels:<br>        app: ratelimit<br>    spec:<br>      containers:<br>        - name: ratelimit<br>          image: envoyproxy/ratelimit:v1.4.0<br>          command: [&quot;ratelimit&quot;]<br>          env:<br>            # - name: LOG_LEVEL<br>            #   value: debug<br>            - name: REDIS_SOCKET_TYPE<br>              value: tcp<br>            - name: REDIS_URL<br>              value: redis-service.marin3r-system.svc.cluster.local:6379<br>            - name: USE_STATSD<br>              value: &quot;false&quot;<br>            - name: RUNIME_ROOT<br>              value: /srv/runtime_data/current<br>            - name: RUNTIME_SUBDIRECTORY<br>              value: &quot;/&quot;<br>            - name: RUNTIME_IGNOREDOTFILES<br>              value: &quot;true&quot;<br>          ports:<br>            - name: http<br>              containerPort: 8080<br>            - name: grpc<br>              containerPort: 8081<br>            - name: debug<br>              containerPort: 6070<br>          resources: {}<br>          volumeMounts:<br>            - mountPath: /srv/runtime_data/current/config<br>              name: runtime-config<br>      volumes:<br>        - name: runtime-config<br>          configMap:<br>            name: ratelimit-config<br>---<br>apiVersion: v1<br>kind: Service<br>metadata:<br>  name: ratilimit-service<br>  namespace: marin3r-system<br>  labels:<br>    app: ratelimit<br>spec:<br>  type: ClusterIP<br>  ports:<br>    - name: ratelimit-8080<br>      port: 8080<br>      targetPort: 8080<br>    - name: ratelimit-8081<br>      port: 8081<br>      targetPort: 8081<br>    - name: ratelimit-6070<br>      port: 6070<br>      targetPort: 6070<br>  selector:<br>    app: ratelimit<br>EOF</pre><p><em>Note that the</em>REDIS_URL<em> env variable receives the Redis service address to allow the rate limit application to connect to the database.</em></p><ul><li>We need to edit the EnvoyConfig CR to define the rate limit configurations, here we are rate limiting any incoming request to our REST API application.</li></ul><pre>cat &lt;&lt;&#39;EOF&#39; | kubectl apply -f -<br>apiVersion: marin3r.3scale.net/v1alpha1<br>kind: EnvoyConfig<br>metadata:<br>  name: httpbin<br>  namespace: rest-application<br>spec:<br>  nodeID: httpbin<br>  serialization: yaml<br>  envoyResources:<br>    clusters:<br>      - name: httpbin<br>        value: |<br>          name: httpbin<br>          connect_timeout: 2s<br>          type: STRICT_DNS<br>          lb_policy: ROUND_ROBIN<br>          load_assignment:<br>            cluster_name: httpbin<br>            endpoints:<br>              - lb_endpoints:<br>                  - endpoint:<br>                      address:<br>                        socket_address:<br>                          address: 127.0.0.1<br>                          port_value: 8080<br>      - name: ratelimit<br>        value: |<br>          name: ratelimit<br>          type: STRICT_DNS<br>          connect_timeout: 2s<br>          lb_policy: ROUND_ROBIN<br>          http2_protocol_options: {}<br>          load_assignment:<br>            cluster_name: ratelimit<br>            endpoints:<br>              - lb_endpoints:<br>                - endpoint:<br>                    address:<br>                      socket_address: { address: ratilimit-service.marin3r-system.svc.cluster.local, port_value: 8081 }<br>    listeners:<br>      - name: http<br>        value: |<br>          name: http<br>          address:<br>            socket_address:<br>              address: 0.0.0.0<br>              port_value: 8443<br>          filter_chains:<br>            - filters:<br>              - name: envoy.http_connection_manager<br>                typed_config:<br>                  &quot;@type&quot;: type.googleapis.com/envoy.config.filter.network.http_connection_manager.v2.HttpConnectionManager<br>                  stat_prefix: ingress_http<br>                  route_config:<br>                    name: local_route<br>                    virtual_hosts:<br>                      - name: httpbin<br>                        domains: [&quot;*&quot;]<br>                        routes:<br>                          - match:<br>                              prefix: &quot;/&quot;<br>                            route:<br>                              cluster: httpbin<br>                              rate_limits:<br>                                - stage: 0<br>                                  actions:<br>                                    - {generic_key: {&quot;descriptor_value&quot;: &quot;slowpath&quot;}}<br>                  http_filters:<br>                    - name: envoy.rate_limit<br>                      config:<br>                        domain: httpbin<br>                        stage: 0<br>                        rate_limit_service:<br>                          grpc_service:<br>                            envoy_grpc:<br>                              cluster_name: ratelimit<br>                            timeout: 2s<br>                    - name: envoy.router<br>EOF</pre><ul><li>If we curl the REST API application host more than one time in an interval of a minute, we will be getting a response 429 - Too many request</li></ul><pre>for i in {1..3}; do curl -I &quot;http://$(oc get route httpbin -o json | jq  -r .spec.host)&quot;; done;</pre><h3>Conclusion</h3><p>In conclusion, I want to mention the importance of adding rate limits to your applications, not only to protect them against attacks but also to prevent resources from being overloaded with unwanted requests.</p><p>Envoy is a really powerful and customizable proxy service that allows you to cover any potential requirements that your application may require to include rate limit to its routes and leveraging MARIN3R Operator to deploy and dynamically set any defined configurations will eliminate any additional work required for maintaining your services protected.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5112858871be" width="1" height="1" alt=""><hr><p><a href="https://itnext.io/rate-limiting-with-marin3r-operator-5112858871be">Rate limiting with MARIN3R Operator</a> was originally published in <a href="https://itnext.io/">ITNEXT</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[TDD: Factory generator with factory.ts and faker.js to ease your testing in TypeScript applications]]></title>
            <link>https://jaferson123.medium.com/tdd-factory-generator-with-factory-ts-and-faker-js-to-ease-your-testing-in-typescript-applications-5afdd45e6e8?source=rss-d694c37534bf------2</link>
            <guid isPermaLink="false">https://medium.com/p/5afdd45e6e8</guid>
            <category><![CDATA[javascript]]></category>
            <category><![CDATA[mock-data]]></category>
            <category><![CDATA[unit-testing]]></category>
            <category><![CDATA[typescript]]></category>
            <dc:creator><![CDATA[Jose Silva]]></dc:creator>
            <pubDate>Wed, 17 Apr 2019 15:24:52 GMT</pubDate>
            <atom:updated>2019-04-17T15:24:52.034Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MP6LSjmnR0fStKToE-s8eg.jpeg" /><figcaption><a href="https://xmpro.com/xmpro-joins-industrial-internet-consortium-smart-factory-machine-learning-testbed/smart-factory-ml-testbed_mini/">https://xmpro.com/xmpro-joins-industrial-internet-consortium-smart-factory-machine-learning-testbed/smart-factory-ml-testbed_mini/</a></figcaption></figure><p>When writing unit testing it is alway a good practice to mock the database with test data to achieve the <a href="https://github.com/ghsukumar/SFDC_Best_Practices/wiki/F.I.R.S.T-Principles-of-Unit-Testing">F.I.R.S.T principles of unit testing</a>.</p><p>With only a few models in an application it is not too difficult to mock the necessary test data, but when your application grows it starts to become unmanageable, for example can you image mocking data for an application with over 200 models?</p><p>A way of avoiding all the tedious work of generating all data required to test a certain method or unit of your application is taking advantage of<strong> factories</strong>, which by definition are the responsible for “manufacturing” data when needed.</p><h3><strong>Making factories generator with TypeScript</strong></h3><p>While working on a TypeScript project I came across a library called <a href="https://www.npmjs.com/package/factory.ts"><strong>factory.ts</strong></a><strong> </strong>to generate model instances by interface or type definition.</p><p>Let say we have an interface <em>IDepartment</em> with two properties <em>id</em> and <em>name </em>and we want to create a factory for our department interface, the only two things we need to do is call the method <em>makeFactory();</em> from the <em>factory.ts</em> library and then return an implementation of <em>IDepartment</em>.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c1641ff97a91215bfdbadc6cd5712758/href">https://medium.com/media/c1641ff97a91215bfdbadc6cd5712758/href</a></iframe><p>So after creating the department factory you can reference the instance elsewhere on the application…</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/be5796d8e9aa5843ed988c48f88b823c/href">https://medium.com/media/be5796d8e9aa5843ed988c48f88b823c/href</a></iframe><p>As the snippet above demonstrates every time an instance of <em>IDepartment</em> is required, you just need to invoke the <em>build();</em> or <em>buildList();</em> method from department factory and an instance will be provided, you can also easily override value of the interface properties when instantiating the factories by passing the desired properties as a parameter on the <em>build()</em> method.</p><h3>Faking the factories</h3><p>Generating random data for factories is great when it comes to test the performance of your code with big amount of data or simple populate a list with some random objects.</p><p>Going back to our example, we can easily generate a list of departments by assigning random data to the properties of the <em>IDepartment</em> interface. There are a ton of libraries that help creating random data.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/13699026db424ddde92fd38aadcdfac4/href">https://medium.com/media/13699026db424ddde92fd38aadcdfac4/href</a></iframe><p>Notice that <em>id</em> and <em>name</em> properties are generated randomly by the library called<strong> </strong><a href="https://github.com/marak/Faker.js/"><strong><em>faker.js</em></strong></a><em>.</em> For more details about this library check out the documentation and its API which is very powerful.</p><h3>Testing with factories</h3><p>Once the factories are created, it is extremely easy to mock model instances to start testing our code.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/bd6e1db60524d42f933920f3316cd22d/href">https://medium.com/media/bd6e1db60524d42f933920f3316cd22d/href</a></iframe><p>The example above shows that with fewer lines of code, instances of <em>IDepartment</em> are provided to test the method <em>getDepartment();</em> avoiding all hassle of creating the instances manually.</p><h3><strong>Conclusion</strong></h3><p>The example used in this blog was just to demonstrated the usage of creating factories, and indeed for small projects it can be a bit overwhelming but as the application grows and unit testing is taking seriously on the project, the creation of factories comes in very handy, saving a lot of time and surely making testing easier.</p><ul><li><a href="https://github.com/marak/Faker.js/">Marak/faker.js</a></li><li><a href="https://github.com/willryan/factory.ts/tree/feature/async-factories">willryan/factory.ts</a></li><li><a href="https://github.com/ghsukumar/SFDC_Best_Practices/wiki/F.I.R.S.T-Principles-of-Unit-Testing">ghsukumar/SFDC_Best_Practices</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5afdd45e6e8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Memory allocation in JavaScript — Eager vs Lazy allocation approach]]></title>
            <link>https://jaferson123.medium.com/memory-allocation-in-javascript-eager-vs-lazy-allocation-approach-28885d74c4a2?source=rss-d694c37534bf------2</link>
            <guid isPermaLink="false">https://medium.com/p/28885d74c4a2</guid>
            <category><![CDATA[computer-science]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[javascript]]></category>
            <category><![CDATA[web-development]]></category>
            <dc:creator><![CDATA[Jose Silva]]></dc:creator>
            <pubDate>Mon, 16 Apr 2018 09:25:06 GMT</pubDate>
            <atom:updated>2021-05-11T09:58:41.105Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tzvLpenUERVt9kGE4sFMRw.jpeg" /><figcaption>Image Credit: TobiasD / Pixapay</figcaption></figure><p>Memory allocation is the process of reserving complete or partial space (physically or virtual) on the computer to execute an application. The management of memory on a program if not handled in the right way can make the application slow or even stop working and one of the reasons for that to happen is known in computer science as Memory Leak.</p><blockquote>Memory leak occurs when the application allocates spaces in memory and does not release them after it be used, then the application ends up running out of memory and crashing.</blockquote><p>Many programming languages have their approach to prevent memory leaking to happen, in <em>JavaScript</em> the lifetime of an object in memory is managed by the garbage collector. The garbage collector is a process that runs periodically and frees up space in memory that is not associated to any reference.</p><p>In computing terms the words <em>lazy</em> and <em>eager </em>allocation<em> </em>refer<em> </em>to when an application requests memory from the system and how much it requests up-front.</p><p><strong>Eager Allocation </strong>is when the application requires the needed space up-front which can cause overallocation if the system does not exactly know how much memory will be used, but on the other hand the required space is already allocated to be used whenever its needed.</p><p>As oppose to eager allocation, <strong>Lazy Allocation</strong> requires space in memory in the run-time (on demand), which costs less to the system as memory is only allocated when needed, but the down side of it is when the application is running with low spare memory and a portion of code needs more memory to continue executing, the application will probably crash.</p><p>To illustrate the case let’s suppose we have a program that allocates 100 positions in an array and multiply the first 5 elements by 5.</p><p>With an <em>eager allocation</em> approach the application will allocate the 100 spaces first and then multiply the first 5 elements by 5.</p><pre><em>var array = [];<br>var maxElement = 5;</em></pre><pre><em>for (i=0; i &lt; 100; i++) array[i] = i; </em></pre><pre>for (i=0; i &lt; <em>maxElement</em>; i++) array[i] = i * 5;</pre><pre><em>//output <br>// =&gt; [0, 5, 10, 15, 20, </em>5, 6, 7, 8, 9, ...<em>]</em></pre><p>With a <em>lazy allocation</em> approach the application will allocate and multiply over each iteration through the array.</p><pre><em>var array = [];<br>var maxElement = 5;</em></pre><pre><em>for (i=0; i &lt; 100; i++) {<br></em>  if (array.length &gt;= maxElement) <em>break</em>;<br>    array[i] = i;<br>    array[i] = array[i] * 5;<em><br>}</em></pre><pre><em>//output <br>// =&gt; [0, 5, 10, 15, 20]</em></pre><h3><strong>Conclusion</strong></h3><p>In conclusion an <em>eager</em> evaluation will be more efficient when the application uses all the allocated resource for that portion of an expression to run, whether a <em>lazy </em>evaluation will suit better when the application is not aware of the required space of memory to execute.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=28885d74c4a2" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>