<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Yad2 Tech - Medium]]></title>
        <description><![CDATA[Yad2 is Israel’s largest second-hand website. From software to infrastructure this is our platform to share the insights we glean on a daily basis. - Medium]]></description>
        <link>https://medium.com/yad2-tech?source=rss----b4c45608d095---4</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>Yad2 Tech - Medium</title>
            <link>https://medium.com/yad2-tech?source=rss----b4c45608d095---4</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 04:33:32 GMT</lastBuildDate>
        <atom:link href="https://medium.com/feed/yad2-tech" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Getting the most out of Open Telemetry with manual instrumentation]]></title>
            <link>https://medium.com/yad2-tech/getting-the-most-out-of-open-telemetry-with-manual-instrumentation-7074744cf1d2?source=rss----b4c45608d095---4</link>
            <guid isPermaLink="false">https://medium.com/p/7074744cf1d2</guid>
            <category><![CDATA[jaeger]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[opentelemetry]]></category>
            <category><![CDATA[tracing]]></category>
            <category><![CDATA[observability]]></category>
            <dc:creator><![CDATA[Maorgoldberg]]></dc:creator>
            <pubDate>Sun, 10 Oct 2021 08:00:57 GMT</pubDate>
            <atom:updated>2021-10-10T08:00:56.925Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_2cXGOyu8uITHE5Rk13vMA.jpeg" /></figure><p>Implementing observability within your application, whether from the start or to improve legacy insight, is a difficult task. There are many ways to auto-instrument specific parts of your code now, thanks to open telemetry. While this type of instrumentation has its uses and can help you get started with tracing, it is somewhat limited in terms of providing specialized information.</p><p>Manual instrumentation allows us to take tracing into our own hands and choose which parts of our code we want to see. This type of instrumentation has its drawbacks, but proper implementation can help you get the most out of your tracing data.</p><p>This post will look at how to use Open Telemetry for Javascript to implement manual traces within your node applications. We’ll go over the fundamentals of manual tracing and the tools available; this will assume a basic understanding of tracing concepts and Open Telemetry.</p><h3>Tracing Weather</h3><p>Our demo app is a fork of Brandon Morell’s weather app. To demonstrate its capabilities, this app was integrated with basic manual tracing. A few changes have been made, such as the addition of an in-memory SQL database to simulate database calls. This project’s code can be found <a href="https://github.com/maorgoldberg/simple-nodejs-weather-app">here</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*o-1A0x7DIAupE78MMGuUjA.png" /><figcaption>Application Architecture</figcaption></figure><p>This application sends tracing data to the most recent Jaeger in a single Docker image. You only need to run the start script and the all-in-one image to get started on the project.</p><h3>Building Our Tracer</h3><p>To generate tracing data, we must first set up a trace provider. This provider has multiple span processors, but we will only use one. Because we are working with express and want to trace our backend environment, we will initialize the node trace provider.</p><pre>const provider = new NodeTracerProvider({</pre><pre>    resource: new Resource({</pre><pre>        [SemanticResourceAttributes.SERVICE_NAME]: &#39;Weather-app&#39;</pre><pre>    })</pre><pre>});</pre><p>Next, we need to configure our exporter options so open telemetry knows where to send our telemetry data.</p><pre>const options = {</pre><pre>    serviceName: &#39;Weather-app&#39;,</pre><pre>    endpoint: &#39;http://localhost:14268/api/traces&#39;</pre><pre>}</pre><pre>const exporter = new JaegerExporter(options);</pre><pre>const spanProcessor = new BatchSpanProcessor(exporter);</pre><p>Finally, we need to add our span processor and register our tracer so it can be exported and imported to our main server code.</p><pre>provider.addSpanProcessor(spanProcessor);</pre><pre>provider.register();</pre><pre>module.exports = opentelemetry.trace.getTracer(&#39;Weather-app&#39;);</pre><h3>Adding Manual Traces (finally…)</h3><p>Finally, the exciting part of tracing: incorporating our own traces into the application! First, I’d like to go over some general concepts and functions that you’ll need to be familiar with, and then we’ll look at how they apply to our weather app.</p><h3>Accessing the current span</h3><p>There are times when we would like access to the span of an HTTP request, which we could automatically trace. This situation is less relevant in this project because auto-instrumentation is not included. However, this is an important concept that I believe should be addressed. Fortunately, this is a very simple task; all we have to do is call ‘getSpan’ and pass it the active context.</p><pre><br>import { context, trace } from ‘<a href="http://twitter.com/opentelemetry/api">@opentelemetry/api</a>’;<br>const span = trace.getSpan(context.active());<br></pre><p>The functions used to access the spans and the context after creation are not related to the tracer and are instead part of the API.</p><h3>Setting a new current span</h3><pre><br>import { context, trace } from ‘<a href="http://twitter.com/opentelemetry/api">@opentelemetry/api</a>’;<br>const workSpan = tracer.startSpan(‘Work’);</pre><pre>work.doWork()<br>workSpan.end()</pre><p>Assume we wanted to measure the length of our workday as part of a task. We could start a new work span and end it once our function was called. This would work, but our doWork method would have no access to our workSpan and would be unable to add attributes. Furthermore, obtaining the current span within this method would result in the return of the parent span! So, what are we to do? We must inform our code that our current span is the workSpan. To accomplish this, we first make it the desired span in our context object and then use ‘context.with’ to create a closure around the ‘doWork’ method. The ‘getSpan’ method within this closure will now return ‘race.’</p><pre><br>import { context, trace } from ‘<a href="http://twitter.com/opentelemetry/api">@opentelemetry/api</a>’;<br><br>const workSpan = tracer.startSpan(“Work”);<br>context.with(trace.setSpan(context.active(), workSpan), () =&gt; {<br> // now returns the work span<br> console.log(trace.getSpan(context.active()));<br> work.doWork()<br>});</pre><p>This method of wrapping method calls is important because we want the application code to be able to assume that the current span is correct at all times.</p><h3>Setting attributes</h3><p>It is critical to have descriptive attributes when filtering through traces or analyzing specific spans. Here’s an example of how to create a trace with attributes:</p><pre><br>import * as api from ‘<a href="http://twitter.com/opentelemetry/api">@opentelemetry/api</a>’;<br>const parentSpan = tracer.startSpan(‘parent’);<br>api.trace.setSpan(api.context.active(), parentSpan);<br>const span = tracer.startSpan(‘handleRequest’, {<br> kind: api.SpanKind.CLIENT, // server<br> attributes: {<br> “http.method”: “GET”,<br> “http.flavor”: “1.1”,<br> “http.url”: “<a href="https://example.com:8080/project/123/list/?page=2">https://example.com:8080/project/123/list/?page=2</a>&quot;,<br> “net.peer.ip”: “192.0.2.5”,<br> “http.status_code”: 200,<br> “http.status_text”: “OK”},<br> });<br>// In addition to the standard attributes, custom attributes can be added as well.<br>span.setAttribute(“list.page_number”, 2);<br>// To avoid collisions, always namespace your attribute keys using dot notation.<br>span.setAttribute(“project.id”, 2);<br>// attributes can be added to a span at any time before the span is finished.<br>span.end()<br></pre><h3>How This Fits Into Our App</h3><p>Now that we’ve covered the fundamentals of manual tracing and its implementation, we can look at how we addressed these issues in our weather application.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GZ6PGF4FDFD7OKdkNkbTEw.png" /></figure><p>First, because we are not using any automatic instrumentation, we will create a new POST span with our imported tracer. After we’ve created this new span, we’ll use the ‘context.with’ method to wrap our code. This is to avoid the premature closure of our post span.</p><p>Next, we check our database to see if the user has already searched for the specified city; obviously, this is something we want to be traced, so we open a new span. We can see these actions in our spans after checking the database and responding appropriately if the city is present or not.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FTQfmY1RZ3xWJ3ON_31TDw.png" /></figure><p>To allow our callback function access to the correct parent span, we use the same code shown in setting a new current span above in our call to the openweather API.</p><h3>What We See In Jaeger</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3gLjEGw2yRKnBk-2ea7j3w.png" /></figure><p>Loading the trace data into our Jaeger UI reveals that the biggest bottleneck in our application is our call to open weather, while our database calls are negligible (as one would expect from an in-memory database…). We can go deeper into the data to see the attributes we’ve assigned to our spans.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8Gw9nL8PNrJaFNW84S_qAA.png" /></figure><p>While the app is impractical in practice and does not benefit greatly from these traces, the methods by which they are implemented can be used in a variety of applications.</p><h3>Conclusion</h3><p>In this article, we looked at how to use manual tracing to improve our application’s telemetry. Where auto instrumentation is insufficient, this type of instrumentation can be used to fine-tune your instrumentation efforts. As open telemetry evolves, so will manual instrumentation methods; however, this was an overview of the concepts as well as implementation and may still be useful as open telemetry matures.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7074744cf1d2" width="1" height="1" alt=""><hr><p><a href="https://medium.com/yad2-tech/getting-the-most-out-of-open-telemetry-with-manual-instrumentation-7074744cf1d2">Getting the most out of Open Telemetry with manual instrumentation</a> was originally published in <a href="https://medium.com/yad2-tech">Yad2 Tech</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Thinking GitOps — Pingdom Controller]]></title>
            <link>https://medium.com/yad2-tech/thinking-gitops-pingdom-controller-6bc2821f66ec?source=rss----b4c45608d095---4</link>
            <guid isPermaLink="false">https://medium.com/p/6bc2821f66ec</guid>
            <category><![CDATA[pingdom]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[flux]]></category>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[gitops]]></category>
            <dc:creator><![CDATA[Ido Braunstain]]></dc:creator>
            <pubDate>Sun, 13 Jun 2021 09:33:18 GMT</pubDate>
            <atom:updated>2021-06-13T09:33:18.189Z</atom:updated>
            <content:encoded><![CDATA[<h3>Thinking GitOps — Pingdom Controller</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/845/1*iLBo4CSH_a8SwCvJ4Vi0OA.png" /></figure><p>One of my first tasks at Yad2 was to redesign the current CI/CD pipelines which were pretty straightforward at the time.<br>We had a single On-premise Gitlab server that was responsible for:</p><ul><li>Source code</li><li>Docker image registry</li><li>CI pipelines</li><li>CD pipelines</li></ul><p>Not a high availability solution, to say the least.<br>This task was my chance to try something new and exciting, that’s why I decided to learn about GitOps.</p><blockquote>GitOps is a way to do Kubernetes cluster management and application delivery. It works by using Git as a single source of truth for declarative infrastructure and applications.</blockquote><p>We decided to try out <a href="https://fluxcd.io/">Flux</a> by <a href="https://www.weave.works/">Weaveworks</a> for the implantation.<br>Let’s go over the four pillars of GitOps from Yad2 point of view:</p><h4>1. The entire system is described declaratively</h4><p>We started to recreate the entire Kubernetes’ manifests and store them in a simple-to-navigate layout inside git.</p><pre>.<br>├── releases<br>│   ├── dev<br>│   ├── dr<br>│   ├── prod<br>└── workloads<br>    ├── dev<br>    ├── dr<br>    └── prod</pre><p>For the first time, we can easily see which version is running and what the current configuration is.<br>Communication with the Kubernetes API is declarative so, we don’t do things more than once. For example, deploying the complex Prometheus Operator chart, After testing the single helmreleasefile in our dev environment releasing it to production was a simple copy operation.</p><h4>2. The canonical desired system state versioned in Git</h4><p>Without a doubt, Git is the most widely used modern version control system in the world today. Storing our infrastructure in Git equips us with some useful features, such as:</p><ul><li>Audit</li><li>Pull Request</li><li>History</li><li>Flexibility</li><li>Rollback</li></ul><h4>3. Approved changes that can be automatically applied to the system</h4><p>After storing the state of our entire cluster in Git, the next step was to ensure that state changes in Git are automatically rolled out to the cluster. We don’t need to further complicate our CI process or provide any more credentials, as Flux does that for us. Flux runs on our cluster and pulls the changes from Git to be deployed straight to k8s.</p><h4>4. Software agents ensure correctness and alert on divergence</h4><p>Any change must go through git so, we were able to implement checks and catch any error at an early stage or even prevent it from applying to the cluster.<br>Simple example — YAML lint.</p><p>After the “why” and “how” we moved to the GitOps methodology, I’d like to introduce you to a simple state of mind that changed through our migration process.</p><h3>The problem</h3><p>We are using <a href="https://www.pingdom.com/">Pingdom</a> to monitor our external endpoints.<br>For each external endpoint, we need to create a Check within the Pingdom website.<br>Creating aCheck looks like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/676/1*XdIUIyV27_Pu1Pvhs67ymA.png" /></figure><p>As the number of endpoints increases, the process of creating a new Check and managing those endpoints becomes a toil, and toil should be eliminated.<br>The first step to solving this is writing code that automates our manual actions. We still had to think about how to solve this in a way that fits our existing work methodology — GitOps.<br>We thought it would be a great opportunity to create a CRD but, the overhead of managing a CRD is not justified in this case.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/588/1*mwcscUhfttnj8oiPD0Fu_w.png" /></figure><h3>The solution</h3><p>As a result, we decided to create a simple controller called Pingdom-controller. We will make use of the already created Ingress resource for each endpoint.<br>Each Ingress event will trigger an immediate Pingdom-controller reconcile loop cycle. Additionally, the controller will act according to a specific set of custom Annotations.</p><p>For example, adding just two annotations to an existing ingress will causePingdom-controller to create a matching Check:</p><pre>apiVersion: extensions/v1beta1<br>kind: Ingress<br>metadata:<br>  annotations:<br>    nginx.ingress.kubernetes.io/rewrite-target: /<br>  <strong>  pingdom.controller.yad2/apply: &quot;true&quot;<br>    pingdom.controller.yad2/resolution: &quot;1&quot;</strong><br>  name: my-service <strong># Check name</strong><br>spec:<br>  rules:<br>    - host: my-service.com <strong># Check host target</strong><br>      http:<br>        paths:<br>          - backend:<br>              serviceName: service<br>              servicePort: http<br>            path: /</pre><p>In order to avoid further burdening to the Kubernetes API servers (control plane), we opted to use “Informers and caching” instead of a watch command.<br>Scaffold for the reconcile loop:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/8c70d30076b453e7bb27b4abb38bc861/href">https://medium.com/media/8c70d30076b453e7bb27b4abb38bc861/href</a></iframe><p>We are using an Informer for almost every use case that requires retrieving events from Kubernetes which gives us: in-memory caching and fast, indexed lookup of objects by name or other properties in-memory.<br>A controller that accesses the API server using a watch mechanism every time it needs an object creates a high load on the system. In-memory caching using informers is the solution to this problem. Moreover, informers can react to changes of objects nearly in real-time instead of requiring polling requests.</p><p>In this article, I showed you an example of how GitOps has changed the way we work and simplified our day-to-day toils.<br>The entire project is available at Github: <a href="https://github.com/axel-springer-kugawana/y2_pingdom_controller">https://github.com/axel-springer-kugawana/y2_pingdom_controller</a>, feel free to use it.<br>Contributions of any kind are welcome!</p><h3>References</h3><ul><li><a href="https://www.oreilly.com/library/view/programming-kubernetes/9781492047094/">Programming Kubernetes</a></li><li><a href="https://www.weave.works/technologies/gitops/">GitOps what you need to know</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6bc2821f66ec" width="1" height="1" alt=""><hr><p><a href="https://medium.com/yad2-tech/thinking-gitops-pingdom-controller-6bc2821f66ec">Thinking GitOps — Pingdom Controller</a> was originally published in <a href="https://medium.com/yad2-tech">Yad2 Tech</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>