<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Emre Önal on Medium]]></title>
        <description><![CDATA[Stories by Emre Önal on Medium]]></description>
        <link>https://medium.com/@nemre?source=rss-a7a80c91eaa1------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*za0AZl8mnlS2x12Xd3XsOA.jpeg</url>
            <title>Stories by Emre Önal on Medium</title>
            <link>https://medium.com/@nemre?source=rss-a7a80c91eaa1------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 09:23:48 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@nemre/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Is Your Go Application Really Using the Correct Number of CPU Cores?]]></title>
            <link>https://nemre.medium.com/is-your-go-application-really-using-the-correct-number-of-cpu-cores-20915d2b6ccb?source=rss-a7a80c91eaa1------2</link>
            <guid isPermaLink="false">https://medium.com/p/20915d2b6ccb</guid>
            <category><![CDATA[gomaxprocs]]></category>
            <category><![CDATA[cpu]]></category>
            <category><![CDATA[automaxprocs]]></category>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[kubernetes]]></category>
            <dc:creator><![CDATA[Emre Önal]]></dc:creator>
            <pubDate>Mon, 03 Jul 2023 21:28:54 GMT</pubDate>
            <atom:updated>2023-07-03T21:28:54.167Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gFYwi8ejn4he4XZGUH-pMw.jpeg" /></figure><p>In the world of modern cloud-native languages, Go has gained significant popularity for its efficiency and concurrency support through goroutines. However, when it comes to utilizing the appropriate number of CPU cores, Go applications face some challenges. This article explores the underlying issues and sheds light on why Go applications may fail to leverage the correct number of CPU cores, particularly in containerized environments like Kubernetes.</p><h4>Go’s GMP Scheduler</h4><p>Go employs a unique scheduling model called GMP, where G represents goroutines, M represents kernel-level threads, and P represents available CPU cores. Multiple goroutines can share a single thread (M), which needs to be bound to a specific P during execution. If the number of CPUs perceived by the operating system differs from the number of available CPUs perceived by a Go process, even binding an M to a specific P may not guarantee its execution. Therefore, the ability to obtain an accurate count of available CPU cores directly affects Go’s scheduling efficiency.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1tp-A27b6QcboVl4nDqmSg.png" /></figure><h4>Resource Limits in Kubernetes</h4><p>Suppose a user sets resource limits for a Go application in a Kubernetes environment using the following specification:</p><pre>spec:<br>  containers:<br>  - name: &quot;my-go-app&quot;<br>    resources:<br>      limits:<br>        cpu: &quot;4&quot;</pre><p>One might assume that Go, being a popular language in the cloud-native era, would have built-in support for Kubernetes and automatically recognize these limits. However, that assumption is not entirely accurate. Let’s conduct a small experiment:</p><p>If the GOMAXPROCS environment variable is not explicitly set when starting the Go process, it defaults to the output of the runtime.NumCPU() function, which represents the number of available CPU cores (i.e., the value of P). However, adding a print statement for NumCPU reveals that it actually outputs the number of CPUs on the node, regardless of the limits.cpu value.</p><pre>package main<br><br>import &quot;runtime&quot;<br><br>func main() {<br>  print(runtime.NumCPU()) // equals the number of CPUs of the node<br>}</pre><h4>Limitations of runtime.NumCPU()</h4><p>On Linux, the runtime.NumCPU() function relies on the sched_getaffinity system call, which only considers CPU core bindings and doesn’t take into account CPU resource restrictions imposed by cgroups in container environments.</p><p>For instance, in Docker, the --cpuset-cpus flag defines the CPU core numbers available for a container at runtime, while restricting CPU resource usage primarily relies on --cpus=. Only the former (cpuset) can be recognized by sched_getaffinity.</p><p>For more information, refer to the Docker <a href="https://docs.docker.com/config/containers/resource_constraints/#configure-the-default-cfs-scheduler">documentation</a> on resource constraints.</p><h4>Accurate Calculation of CPU Cores</h4><p>Fortunately, there is a Go <a href="https://medium.com/github.com/uber-go/automaxprocs">library</a>, that accurately calculates GOMAXPROCS by reading the cgroup file system.</p><p>This library supports both cgroup v1 and v2. In cgroup v1, it reads the cpu.cfs_quota_us and cpu.cfs_period_us files and calculates their quotient. Typically, these files are located under /sys/fs/cgroup/cpu/ (automaxprocs retrieves the mount information to determine the actual location). In cgroup v2, it reads the corresponding fields representing quota and period in the cpu.max file and calculates their quotient.</p><p>Since the division result may not be an integer, an additional step of rounding down is performed.</p><h4>Challenges Beyond Go</h4><p>It’s important to note that Go is not the only language facing difficulties in recognizing the available CPU cores accurately within container environments. Nginx, Envoy, and other frameworks also struggle with recognizing cgroup configurations. However, it’s been reported that Rust and Java (OpenJDK implementation) have dedicated solutions to handle cgroup configurations.</p><p>If your application meets the following criteria: Performs compute-intensive tasks or has critical business logic executed within a fixed number of workers. Deployed in a containerized environment. It’s worth investigating whether the framework you are using properly handles cgroup configurations.</p><h4>Performance Impact</h4><p>Let’s examine the benchmark results of the uber automaxprocs library to see the impact of setting the correct GOMAXPROCS configuration on performance.</p><p>Data measured from Uber’s internal load balancer with 200% CPU quota (i.e., 2 cores):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hYE6nmacI-rMetPyugSy8Q.png" /></figure><p>When GOMAXPROCS is increased above the CPU quota, we see P50 decrease slightly, but see significant increases to P99. We also see that the total RPS handled also decreases.</p><h4>Conclusion</h4><p>The mismatch between perceived and actual CPU core availability in container environments poses a challenge for Go applications. Although Go relies on the runtime.NumCPU() function, it doesn’t consider the CPU resource limitations imposed by groups.</p><p>Developers must be aware of this limitation and consider utilizing libraries like <a href="https://github.com/uber-go/automaxprocs">automaxprocs</a> to accurately determine the optimal number of CPU cores for their Go applications. As the cloud-native landscape evolves, addressing these challenges will ensure efficient resource utilization and enhance the performance of Go applications in containerized environments.</p><p><em>Good luck and happy coding!</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=20915d2b6ccb" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Manage ArgoCD Resources Programmatically with Golang ]]></title>
            <link>https://nemre.medium.com/manage-argocd-resources-programmatically-with-golang-5fa825f1f36e?source=rss-a7a80c91eaa1------2</link>
            <guid isPermaLink="false">https://medium.com/p/5fa825f1f36e</guid>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[argo-cd]]></category>
            <category><![CDATA[minikube]]></category>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[kubectl]]></category>
            <dc:creator><![CDATA[Emre Önal]]></dc:creator>
            <pubDate>Wed, 03 May 2023 19:26:36 GMT</pubDate>
            <atom:updated>2023-05-03T19:26:36.062Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EwNqZ01N4ExSM2CQUwro5w.jpeg" /></figure><p>In this article, we will examine how to manage <a href="https://argo-cd.readthedocs.io/">ArgoCD</a> resources with <a href="https://go.dev/">Go</a> for programmatically creating projects, clusters, and destinations with example codes.</p><h4>Prerequisites</h4><ul><li><a href="https://go.dev/doc/install"><strong>Go</strong></a> for developing a resource manager program.</li><li><a href="https://docs.docker.com/desktop/install/mac-install/"><strong>Docker Desktop</strong></a> for virtualization.</li><li><a href="https://minikube.sigs.k8s.io/docs/start/"><strong>Minikube</strong></a> for creating a local Kubernetes cluster.</li><li><a href="https://kubernetes.io/docs/tasks/tools/install-kubectl-macos/"><strong>Kubectl</strong></a> for a Kubernetes command-line tool.</li><li><a href="https://argo-cd.readthedocs.io/en/stable/cli_installation/"><strong>ArgoCD CLI</strong></a> for generating a account token.</li></ul><h4><strong>Steps</strong></h4><p>Open a terminal window and execute the following commands:</p><blockquote><strong>Start Docker Desktop</strong></blockquote><p>This command initiates the Docker engine for virtualization.</p><pre>open -a docker</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xbJlygn6UYMUqHPaCw4FsQ.png" /></figure><blockquote><strong>Create a local Kubernetes cluster</strong></blockquote><p>This command launches the local Kubernetes cluster and adds the Minikube context to the ~/.kube/config file.</p><pre>minikube start</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VeEs45C6-pA4t4ymeNl30g.png" /></figure><blockquote><strong>Switch to Minikube context</strong></blockquote><p>In an environment with multiple clusters, we will switch to the Minikube context using this command.</p><p><em>This step is typically performed automatically after installing Minikube.</em></p><pre>kubectl config use-context minikube</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KAdm_tve7EsFyCgxrKV_tw.png" /></figure><blockquote><strong>Create a namespace</strong></blockquote><p>Namespaces serve as a logical grouping of objects within a Kubernetes cluster. This command establishes a new namespace in the current Minikube Kubernetes context, where the ArgoCD installation will take place.</p><pre>kubectl create namespace argocd</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JtTc1dwepbS-CpQxFQA4zA.png" /></figure><blockquote><strong>Install the latest version of ArgoCD</strong></blockquote><p>This command deploys ArgoCD on the current Minikube Kubernetes context, enabling access to ArgoCD in a local environment.</p><pre>kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NNk8L11Zj_ShCcKJBZ6Z-Q.png" /></figure><blockquote><strong>Check deployments</strong></blockquote><p>Let’s see what happens on the cluster after the ArgoCD installation.</p><pre>kubectl get all -n argocd</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wH5BD8EWbelQl6CZfg6ssQ.png" /></figure><blockquote><strong>Add a new fully authorized account</strong></blockquote><p>While our admin account typically possesses comprehensive permissions, it cannot generate tokens for security reasons.</p><p>Consequently, we must create a fully authorized account to modify ArgoCD resources. This command introduces an account named foo:</p><pre>EDITOR=nano kubectl edit configmap argocd-cm -n argocd</pre><p>Append the following fields to the YAML file in the opened editor and save:</p><pre>data:<br>  accounts.foo: apiKey, login</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GMIMKQ7Ahhk3mUZLjKPM-g.png" /></figure><p>This action successfully generates a new account.</p><p>Next, configure the permissions for the account we created by running the following command to open the editor:</p><pre>EDITOR=nano kubectl edit configmap argocd-rbac-cm -n argocd</pre><p>Append the following fields to the YAML file in the opened editor and save:</p><pre>data:<br>  policy.csv: |<br>    g, foo, role:admin</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*m4-nGU9MkrvEkUh0-qPX8g.png" /></figure><p>We now have a foo account with full permissions.</p><blockquote><strong>Obtain the default admin password</strong></blockquote><p>Upon installation, ArgoCD provides an admin user. To log in to ArgoCD using the admin user, we require the password.</p><pre>kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath=&quot;{.data.password}&quot; | base64 -d; echo</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*B_Crbky2Fa-b0XZ3H9zq1w.png" /></figure><blockquote><strong>Forward argocd-server port to local machine network</strong></blockquote><p>To access ArgoCD from our browser, we must forward the argocd-server port to our local machine. Port forwarding terminates when the terminal closes.</p><p>To execute other commands in the same terminal, add a &amp;to the end of the following command:</p><pre>kubectl port-forward svc/argocd-server -n argocd 8080:443</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cbNdmUgt3Jm9M6qEhJW9rw.png" /></figure><p>This command forwards the ArgoCD server’s port to our local machine’s port 8080. We can now access the ArgoCD web interface by launching a browser and navigating to localhost:8080.</p><blockquote><strong>Generate an account token</strong></blockquote><p>A token is required for ArgoCD resource operations within the Go program. Let’s log in using the ArgoCD CLI:</p><pre>argocd login localhost:8080 --insecure</pre><p>When prompted for a username and password, enter admin the username and the previously acquired password for the admin user.</p><p>After logging in, generate an account token for the foo account with the following command:</p><pre>argocd account generate-token --account foo</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OU_dyBrx0z1zYTVDusOVJg.png" /></figure><h4>Develop Golang Resource Manager Program</h4><p>We have completed the necessary installations. Now, let’s start building our application to manage ArgoCD resources using <strong>Golang</strong>.</p><p><em>In this project, we perform resource operations over </em><a href="https://grpc.io/"><em>gRPC</em></a><em> using the </em><a href="https://github.com/argoproj/argo-cd"><em>ArgoCD Go</em></a><em> library. Because it is more type-safe and fast. You can do the same using REST API. If you want to see the ArgoCD Swagger: </em><a href="https://localhost:8080/swagger-ui"><em>http://localhost:8080/swagger-ui</em></a></p><blockquote><strong>Create project with folder structure</strong></blockquote><p>We may need to manage multiple resources in a project using ArgoCD or another tool. Therefore, we must establish a project structure that supports multiple tools and avoids complexity as it grows.</p><p>We will use the <a href="https://github.com/golang-standards/project-layout/">Standard Go Project Layout</a> for this purpose and include a few useful tools, such as a linter. As an example, we will name our project engine.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d7bde96094c60500476a850ce2ebe21a/href">https://medium.com/media/d7bde96094c60500476a850ce2ebe21a/href</a></iframe><blockquote><strong>Create client connection definitions</strong></blockquote><p>This code defines two structs: Connection and Client. The Connection struct has two fields, Address and Token, both of type string. This struct is used to store information about the connection to an ArgoCD server, such as its address and authentication token.</p><p>The Client struct also has two fields: projectClient and clusterClient. These fields are of type project.ProjectServiceClient and cluster.ClusterServiceClient, respectively. This struct is used to perform operations related to projects and clusters on the ArgoCD API.</p><p>The NewClient function takes a pointer to a Connection struct as input and returns a pointer to a Client struct and an error (if any). This function creates a new ArgoCD API client using the apiclient.NewClient() function from the <a href="https://github.com/argoproj/argo-cd/blob/master/pkg/apiclient/apiclient.go">apiclient</a> package. The values in the Connection struct are used to configure the client’s connection to the ArgoCD server.</p><p>Next, the NewProjectClient() and NewClusterClient() methods are called on the API client object to create clients for projects and clusters. These methods return a context, a client, and an error. The project and cluster clients are then used to initialize the Client struct.</p><p>Overall, this code demonstrates how to set up an ArgoCD API client using Go, allowing you to perform operations related to projects and clusters.</p><p>Here’s the code for the /internal/driver/argocd/client.go file:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/53f7c825eb152320de55160fd6f52a9b/href">https://medium.com/media/53f7c825eb152320de55160fd6f52a9b/href</a></iframe><blockquote><strong>Create cluster resource definitions</strong></blockquote><p>The Client struct has a method called GetClusters() that returns a slice of v1alpha1.Cluster objects and any errors encountered. This method takes no arguments.</p><p>To retrieve the list of available clusters, the method uses the clusterClient field of the Client struct to call the List() method from the cluster package.</p><p>The resulting cl object has an Items field, which contains the list of available clusters. The method returns this Items field as a slice of v1alpha1.Cluster objects, along with a nil error.</p><p>These steps demonstrate how to retrieve the list of available clusters from an ArgoCD server using Go.</p><p>Here’s the code for the /internal/driver/argocd/cluster.go file:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c41d34f98c8c1b77a4053888005f3a7f/href">https://medium.com/media/c41d34f98c8c1b77a4053888005f3a7f/href</a></iframe><blockquote><strong>Create destination resource definitions</strong></blockquote><p>The Client struct has a method called AddDestination() that takes four arguments: projectName, server, namespace, and name.</p><p>The method first calls the GetProject() method from the project package, passing in the projectName argument. This method returns the project with the provided name.</p><p>Next, the GetProject() method updates the project’s destinations by creating a new v1alpha1.ApplicationDestination object and setting its fields to the values of the provided arguments.</p><p>Finally, the method calls the Update() method from the project package, passing in a new ProjectUpdateRequest object containing the updated project. This updates the project’s destinations to include the newly added destination.</p><p>These steps demonstrate how to add a destination to an ArgoCD project using Go.</p><p>Here’s the code for the /internal/driver/argocd/destination.go file:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/01f08ce79cb8055725a39626e0e4a944/href">https://medium.com/media/01f08ce79cb8055725a39626e0e4a944/href</a></iframe><blockquote><strong>Create project resource definitions</strong></blockquote><p>The Client struct, which is used to interact with the ArgoCD API, has several methods defined for managing projects.</p><p>The CreateProject method creates a new project with a given name and returns a pointer to the created AppProject object, along with any errors encountered. This is achieved by calling the Create() method from the project package, which takes a ProjectCreateRequest object containing the AppProject with its Name field set to the provided name.</p><p>The DeleteProject() method deletes a project with a given name by calling the Delete() method from the project package. This method returns an error if the project cannot be deleted.</p><p>The GetProject() method retrieves a project with a given name by calling the Get() method from the project package. This method returns a pointer to the AppProject object and any errors encountered.</p><p>These methods demonstrate how to interact with the ArgoCD project API using Go.</p><p>Here’s the code for the /internal/driver/argocd/project.go file:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ae6d7a08cae32f2012fa50e1b32ed916/href">https://medium.com/media/ae6d7a08cae32f2012fa50e1b32ed916/href</a></iframe><blockquote><strong>Trying to execute some driver resource operations</strong></blockquote><p>In the main() function, we create an connection object with the ArgoCD server’s address and token. Then, using this connection object, we create an client object through which we can interact with the ArgoCD API.</p><p>The code continues by creating a new project named foo and displaying its unique identifier. It then adds a destination to the project and retrieves the project’s namespace field.</p><p>Finally, the code deletes the foo project and retrieves the names of all available clusters.</p><p>Overall, this code performs a set of operations on the ArgoCD server, demonstrating how to interact with it using the provided API methods.</p><p>Here’s the code for the /cmd/engine/main.go file:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6fa948ad4118e1c38a5e40e91dd5129a/href">https://medium.com/media/6fa948ad4118e1c38a5e40e91dd5129a/href</a></iframe><h4>Summary</h4><p>In this project, we utilized some resource definitions as examples. You can perform any ArgoCD resource operations you desire using these code samples.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*us4Ngc4lbX7wbwvVC-62EQ.jpeg" /></figure><p>For example, I am trying these example PoC codes to develop the CD processes of a serverless FaaS project similar to <a href="https://www.openfaas.com/">OpenFaaS</a>, which can also be used for <a href="https://en.wikipedia.org/wiki/Site_reliability_engineering">SRE</a> automations and other purposes.</p><p>You can access the full code <a href="https://gist.github.com/nemre/803e513f8d226d5c3d3d491af7695539"><strong>here</strong></a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5fa825f1f36e" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>