<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by NIKITA VERMA  on Medium]]></title>
        <description><![CDATA[Stories by NIKITA VERMA  on Medium]]></description>
        <link>https://medium.com/@nikitackb456?source=rss-9697c9addf03------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*z3SO2jAd0UMiGyfZxNWWiQ.jpeg</url>
            <title>Stories by NIKITA VERMA  on Medium</title>
            <link>https://medium.com/@nikitackb456?source=rss-9697c9addf03------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 07:02:25 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@nikitackb456/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Chapter 3 2:17AM]]></title>
            <link>https://blog.devops.dev/chapter-3-2-17am-06244313a8a1?source=rss-9697c9addf03------2</link>
            <guid isPermaLink="false">https://medium.com/p/06244313a8a1</guid>
            <category><![CDATA[sre]]></category>
            <category><![CDATA[incident-management]]></category>
            <category><![CDATA[vulnerability-management]]></category>
            <category><![CDATA[metrics]]></category>
            <category><![CDATA[software-engineering]]></category>
            <dc:creator><![CDATA[NIKITA VERMA ]]></dc:creator>
            <pubDate>Thu, 23 Jul 2026 14:45:13 GMT</pubDate>
            <atom:updated>2026-08-20T09:14:01.867Z</atom:updated>
            <content:encoded><![CDATA[<h3>Chapter 3</h3><h3>From Golden Signals to Root Cause — How Modern Engineers Debug Distributed Systems</h3><p>It is <strong>2:17 AM</strong>.</p><p>Your phone vibrates.</p><p>A PagerDuty notification flashes across the screen:</p><blockquote><strong><em>🚨 Critical Alert: Checkout Service — P95 Latency has exceeded the threshold for the last 5 minutes.</em></strong></blockquote><p>Customers are reporting slow checkouts. Orders are failing. Revenue is dropping with every passing minute.</p><p>You open Grafana.</p><p>The dashboard confirms the alert — latency has suddenly spiked.</p><p>CPU utilization looks perfectly normal.</p><p>Memory usage is stable.</p><p>Network traffic hasn’t changed.</p><p>Nothing appears to be obviously broken.</p><p>So where do you begin?</p><p>Many engineers instinctively start clicking through dashboards, scanning logs, or restarting services, hoping something stands out. Sometimes that works. More often, it wastes valuable time while the incident continues to affect users.</p><p>The most effective engineers don’t investigate production systems randomly.</p><p>They follow a structured workflow.</p><p>They start with <strong>metrics</strong> to identify <strong>what</strong> is wrong, use <strong>distributed traces</strong> to understand <strong>where</strong> the problem is occurring, and finally analyze <strong>logs</strong> to determine <strong>why</strong> it happened.</p><p>This investigative workflow is what transforms observability from a collection of monitoring tools into a systematic approach for understanding complex distributed systems.</p><p>In the previous chapter of this series, we explored how to correlate <strong>Jaeger traces</strong> with <strong>Loki logs</strong>, allowing us to move seamlessly from a single request’s execution path to the detailed log entries that explain its behavior. That completed two-thirds of the observability story.</p><p>Now it’s time to introduce the third signal — <strong>metrics</strong>.</p><p>Metrics provide the earliest indication that something in your system is behaving unexpectedly. They answer questions like:</p><ul><li>Is latency increasing?</li><li>Are error rates rising?</li><li>Has request traffic suddenly changed?</li><li>Are resources approaching saturation?</li></ul><p>But metrics alone rarely explain <em>why</em> something is happening.</p><p>That’s where traces and logs complete the investigation.</p><p>In this chapter, we’ll build an end-to-end observability workflow using <strong>Prometheus</strong>, <strong>Grafana</strong>, <strong>Jaeger</strong>, and <strong>Loki</strong>. Instead of creating dashboards for the sake of visualization, we’ll learn how experienced Site Reliability Engineers (SREs) investigate real production incidents by moving naturally from <strong>Golden Signals → Metrics → Traces → Logs → Root Cause</strong>.</p><p>By the end of this chapter, you won’t just know how to create dashboards — you’ll understand how modern engineering teams use observability to debug distributed systems with confidence.</p><p>Let’s begin.</p><h3>A Production Incident: Where Every Investigation Starts</h3><p>Imagine you’re the engineer on call for a large-scale e-commerce platform running on Kubernetes.</p><p>Everything appears normal — until it doesn’t.</p><p>Suddenly, customer support begins receiving complaints that the checkout page is taking significantly longer to load. Some users are able to complete their purchases after several seconds, while others are greeted with timeout errors before the payment is even processed.</p><p>Within moments, your incident management platform triggers an alert.</p><pre>🚨 Severity: Critical</pre><pre>Service: Checkout API</pre><pre>P95 Request Latency: 2.8 s (Threshold: 500 ms)</pre><pre>Error Rate: 12%</pre><pre>Traffic: Normal</pre><pre>Status: Investigating...</pre><p>At first glance, nothing appears unusual.</p><p>The Kubernetes cluster is healthy.</p><p>Pods are running.</p><p>CPU utilization is below 40%.</p><p>Memory consumption is stable.</p><p>No containers have restarted.</p><p>No deployments have occurred recently.</p><p>Everything <em>looks</em> healthy.</p><p>Yet customers continue experiencing failures.</p><p>So where do you begin?</p><p>Do you start reading logs?</p><p>Open Jaeger?</p><p>SSH into a node?</p><p>Restart the application?</p><p>Or simply wait and hope the problem disappears?</p><p>Every experienced Site Reliability Engineer knows that these approaches have one thing in common — they’re based on assumptions rather than evidence.</p><p>Production debugging should never begin with guessing.</p><p>It should begin with <strong>observation</strong>.</p><p>And that is precisely where observability changes the way we think about troubleshooting distributed systems.</p><p>Instead of asking,</p><blockquote><strong><em>“Which service is broken?”</em></strong></blockquote><p>we first ask,</p><blockquote><strong><em>“What is the system trying to tell us?”</em></strong></blockquote><p>The first clues rarely come from log files.</p><p>They come from <strong>metrics</strong>.</p><p>Metrics provide the earliest indication that the behavior of a system has changed. They reveal trends, anomalies, and patterns long before we understand their underlying cause.</p><p>A sudden increase in request latency.</p><p>A growing error rate.</p><p>An unexpected spike in traffic.</p><p>A service approaching resource exhaustion.</p><p>Each of these is a symptom — not the disease.</p><p>Our goal is to identify those symptoms quickly, investigate them systematically, and follow the evidence until we reach the root cause.</p><p>That investigative journey begins with one of the most influential concepts in modern Site Reliability Engineering:</p><p><strong>The Four Golden Signals.</strong></p><h3>The Four Golden Signals: The First Questions Every SRE Asks</h3><p>When an incident occurs, experienced engineers don’t immediately search through logs or inspect application code. They first step back and ask a much more fundamental question:</p><blockquote><strong><em>“What is the system trying to tell me?”</em></strong></blockquote><p>Every production system continuously emits thousands of data points every second. CPU utilization, memory usage, request latency, network throughput, error rates, database queries, cache hits, and countless other signals compete for our attention.</p><p>The challenge isn’t a lack of information.</p><p>It’s knowing <strong>which information actually matters</strong>.</p><p>Over the years, Google Site Reliability Engineers observed that although every application is different, the first stage of investigating almost every production incident can be reduced to just <strong>four fundamental questions</strong>.</p><p>Instead of monitoring hundreds of disconnected metrics, they focused on four high-level signals that quickly reveal whether a service is healthy or drifting toward failure.</p><p>These signals are known as <strong>The Four Golden Signals</strong>.</p><p>Think of them as the vital signs of a distributed system.</p><p>Just as a doctor doesn’t begin a diagnosis by ordering dozens of complex laboratory tests, an experienced engineer doesn’t begin debugging by examining thousands of log entries.</p><p>A doctor first checks the patient’s pulse, blood pressure, temperature, and oxygen levels.</p><p>These measurements don’t reveal the exact disease.</p><p>They reveal <strong>whether something is wrong</strong>.</p><p>The Four Golden Signals serve the same purpose for software systems.</p><p>They don’t immediately tell us why an application is failing.</p><p>Instead, they point us toward <strong>where our investigation should begin.</strong></p><p>Before we investigate our checkout incident, let’s understand what each signal represents and, more importantly, <strong>what question it helps us answer during an outage.</strong></p><h3>1. Latency — Is My System Becoming Slower?</h3><p>Imagine you’re back in the middle of our checkout service incident.</p><p>Customers aren’t reporting complete failures.</p><p>Instead, they’re saying something far more dangerous.</p><blockquote><strong><em>“The website feels slow.”</em></strong></blockquote><p>This single sentence should immediately direct your attention toward one of the most critical indicators of application health:</p><p><strong>Latency.</strong></p><p>Latency measures the amount of time required for a system to process a request and return a response.</p><p>Every interaction a user performs — opening a product page, searching for an item, completing a payment, or downloading an invoice — has an associated response time.</p><p>When latency begins to increase, users notice long before infrastructure begins to fail.</p><p>That’s why latency is often the earliest visible symptom of a production incident.</p><p>However, one common mistake made by new engineers is relying solely on the <strong>average response time</strong>.</p><p>Imagine ten requests complete in <strong>100 milliseconds</strong>, while one request takes <strong>8 seconds</strong>.</p><p>The average latency still appears acceptable.</p><p>Yet the customer who experienced that 8-second delay doesn’t care about the average.</p><p>Their experience was poor.</p><p>For this reason, modern monitoring systems emphasize <strong>percentiles</strong>, particularly <strong>P95</strong> and <strong>P99</strong> latency.</p><p>These metrics answer questions such as:</p><ul><li>What response time do <strong>95%</strong> of users experience?</li><li>How slow are the worst-performing requests?</li><li>Are only a small group of users affected, or is the slowdown widespread?</li></ul><p>This is precisely why our incident alert reported:</p><blockquote><strong><em>P95 Latency: 2.8 seconds</em></strong></blockquote><p>rather than</p><blockquote><strong><em>Average Latency: 450 milliseconds</em></strong></blockquote><p>One number reflects the experience of almost every customer.</p><p>The other can easily hide serious performance problems.</p><h3>💡 Engineering Insight</h3><p>One of the biggest mistakes engineers make during an incident is optimizing for <strong>average performance</strong> instead of <strong>user experience</strong>.</p><p>Imagine an e-commerce platform serving one million requests in an hour. If 950,000 requests complete within 100 milliseconds while the remaining 50,000 requests take over 8 seconds, the average latency may still appear healthy. Yet for those fifty thousand users, the system is effectively unusable.</p><p>This is why modern monitoring platforms emphasize <strong>tail latency</strong> through metrics like <strong>P95</strong> and <strong>P99</strong>. They expose performance problems that averages often conceal, allowing engineers to detect user-impacting regressions before they become widespread outages.</p><h3>Returning to Our Incident</h3><p>At this stage, we’ve learned one important fact.</p><p>The checkout service is slow.</p><p>But knowing that requests are taking longer than expected still doesn’t explain <strong>why</strong>.</p><p>A rise in latency could be caused by numerous factors:</p><ul><li>A sudden increase in user traffic.</li><li>A downstream database responding slowly.</li><li>An external payment gateway experiencing delays.</li><li>Resource exhaustion inside the Kubernetes cluster.</li><li>Network congestion between services.</li></ul><p>Latency tells us <strong>that</strong> something is wrong.</p><p>It doesn’t tell us <strong>what changed</strong>.</p><p>To answer that question, we need another perspective.</p><p>Our next question is simple:</p><blockquote><strong><em>Has the amount of traffic reaching our service changed?</em></strong></blockquote><p>That leads us to the second Golden Signal.</p><h3>Before We Continue…</h3><p>At this point, we’ve answered our first question.</p><p>We know that the checkout service is slower than expected.</p><p>But production incidents are rarely solved with a single metric.</p><p>A rise in latency is only the beginning of the investigation.</p><p>The next step is determining <strong>whether the slowdown is caused by increased demand or by something deeper inside the application itself.</strong></p><p>That’s where the second Golden Signal becomes critical.</p><p>In the next part of this series, we’ll continue our investigation by exploring <strong>Traffic, Errors, and Saturation</strong>, and learn how these signals work together to paint a complete picture of system health before we ever open a log file or inspect a trace.</p><p>Because in modern distributed systems…</p><blockquote><strong><em>Great engineers don’t collect metrics.</em></strong></blockquote><blockquote><strong><em>They ask the right questions.</em></strong></blockquote><p>And every Golden Signal answers one of those questions.</p><h3>What’s Coming Next?</h3><p>In <strong>Part 2</strong>, we’ll continue the same production incident and answer three critical questions:</p><ul><li>📈 Has traffic suddenly increased?</li><li>❌ Are users actually experiencing failures?</li><li>⚙️ Is the system approaching its capacity limits?</li></ul><p>By the end of Part 2, you’ll understand <strong>why the Four Golden Signals form the foundation of every observability strategy</strong> — and why experienced SREs look at them before opening a single log file.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=06244313a8a1" width="1" height="1" alt=""><hr><p><a href="https://blog.devops.dev/chapter-3-2-17am-06244313a8a1">Chapter 3 2:17AM</a> was originally published in <a href="https://blog.devops.dev/">DevOps.dev</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Debugging Like a Detective: A Practical Guide to Correlating Jaeger Traces with Loki Logs]]></title>
            <link>https://medium.com/@nikitackb456/debugging-like-a-detective-a-practical-guide-to-correlating-jaeger-traces-with-loki-logs-08b17cddfd7a?source=rss-9697c9addf03------2</link>
            <guid isPermaLink="false">https://medium.com/p/08b17cddfd7a</guid>
            <category><![CDATA[software-reliability]]></category>
            <category><![CDATA[jaeger]]></category>
            <category><![CDATA[debugging]]></category>
            <category><![CDATA[logs]]></category>
            <category><![CDATA[loki]]></category>
            <dc:creator><![CDATA[NIKITA VERMA ]]></dc:creator>
            <pubDate>Wed, 15 Oct 2025 06:35:29 GMT</pubDate>
            <atom:updated>2025-10-15T06:35:29.522Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*GGid42kCRiWAT1j6" /></figure><blockquote><em>In our last post, we set up OpenTelemetry to generate traces and view them in Jaeger. It’s a great first step, but a trace only tells you the </em>what<em> and </em>where<em> of a problem, not the </em>why<em>. For that, we need logs. In this article, we’ll play detective and learn how to link these two critical signals together. 🕵️‍♀️</em></blockquote><h3><em>The Magic Glue: </em><strong><em>The </em></strong><strong><em>TraceID</em></strong></h3><blockquote><em>The secret to correlating everything is the </em><em>TraceID. When OpenTelemetry starts a trace, it generates a unique ID. The magic happens when we configure our application&#39;s logger to automatically inject this </em><em>TraceID into every log line written during that request. Suddenly, your logs are no longer a sea of disconnected messages; they&#39;re clues tied to a specific case file.</em></blockquote><h4><em>Step 1: Enriching Your Logs</em></h4><blockquote><em>First, let’s update our application. If you’re using a structured logger in Node.js (like Pino or Winston), you can use a simple formatter to grab the active span’s context and add the </em><em>trace_id to your JSON logs.</em></blockquote><pre>// Example with a simple logger middleware<br>const { trace } = require(&#39;@opentelemetry/api&#39;);</pre><blockquote><em>function enrichLogs(req, res, next) { const spanContext = trace.getSpanContext(trace.getActiveSpan()); if (spanContext) { req.log = req.log.child({ trace_id: spanContext.traceId }); } next(); }</em></blockquote><blockquote><em>With this, every log line will now look something like this: </em><em>{&quot;level&quot;:&quot;info&quot;,&quot;time&quot;:167765... ,&quot;msg&quot;:&quot;User checkout failed&quot;,&quot;trace_id&quot;:&quot;a1b2c3d4...&quot;}</em></blockquote><h4><strong><em>Step 2: Setting up Grafana for Correlation</em></strong></h4><blockquote><em>Now, for the “Aha!” moment. In Grafana, you need to have both your Jaeger (or Tempo) and Loki data sources configured. The trick is to configure the Jaeger data source to create a link to your Loki logs.</em></blockquote><blockquote><em>In your Jaeger data source settings, under “Trace to logs”, create a link with the following settings:</em></blockquote><ul><li><strong>Target data source:</strong> Loki</li><li><strong>Query:</strong> &#39;{job=&quot;your-app&quot;} | json | trace_id=&quot;${__trace.traceId}&quot;&#39;</li></ul><blockquote><em>The Workflow in Action 🔍</em></blockquote><blockquote><em>Now, your debugging workflow is transformed:</em></blockquote><ul><li>You spot a slow or failed trace in Jaeger within Grafana’s Explore view.</li><li>You click on a span in the trace.</li><li>You see a new “Logs” button appear in the span details.</li><li>Clicking it instantly runs a Loki query for all log lines that contain that exact TraceID!</li></ul><p>You’ve just jumped from seeing a problem’s effect (the slow trace) to its root cause (the error log) in a single click.</p><p>What’s Next? 🌟Now that we’ve connected traces and logs, the observability trinity is nearly complete. In our next post, we’ll bring in the third signal: metrics. We’ll build a “Golden Signals” dashboard in Grafana and learn how to jump from a spike in your error metrics directly to the traces and logs that caused it.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=08b17cddfd7a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Mastering OpenTelemetry: Instrumenting Your Microservices Like a Pro ️]]></title>
            <link>https://blog.devops.dev/mastering-opentelemetry-instrumenting-your-microservices-like-a-pro-%EF%B8%8F-903a08960fc1?source=rss-9697c9addf03------2</link>
            <guid isPermaLink="false">https://medium.com/p/903a08960fc1</guid>
            <category><![CDATA[otel]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[open-source]]></category>
            <dc:creator><![CDATA[NIKITA VERMA ]]></dc:creator>
            <pubDate>Sat, 16 Aug 2025 07:43:08 GMT</pubDate>
            <atom:updated>2025-09-01T17:58:15.616Z</atom:updated>
            <content:encoded><![CDATA[<p>Microservices are powerful, but debugging them without the right observability tools is like flying blind. Logs, metrics, and traces are your cockpit instruments — and <strong>OpenTelemetry (OTel)</strong> is the standard that brings them all together. 🌍✨</p><p>If Jaeger is the detective board for tracing, OpenTelemetry is the <strong>glue</strong> that connects everything: traces, metrics, and logs.</p><h3>What is OpenTelemetry? 🤔</h3><p>OpenTelemetry (a CNCF project) is an open-source observability framework that provides:</p><ul><li><strong>APIs + SDKs</strong> → to generate telemetry data (traces, metrics, logs).</li><li><strong>Collectors</strong> → to process, transform, and export data.</li><li><strong>Interoperability</strong> → works with Jaeger, Prometheus, Loki, Tempo, Grafana, and more.</li></ul><p>In short: OTel is the <strong>standard language</strong> for observability.</p><h3>Why OpenTelemetry Matters 🚀</h3><p>Here’s why developers (and SREs) love it:</p><ul><li><strong>Vendor-neutral</strong>: Instrument once, export anywhere.</li><li><strong>Polyglot</strong>: Works across Go, Node.js, Python, Java, Rust… you name it.</li><li><strong>Unified Observability</strong>: Instead of separate setups for logs, metrics, and traces, OTel unifies them.</li></ul><p>Think of OTel as the <strong>universal adapter</strong> for your observability stack. 🔌</p><h3>Setting Up OpenTelemetry in Your Microservice</h3><h3>Step 1: Add OpenTelemetry SDK</h3><p>Let’s take a Node.js example:</p><pre>const { NodeTracerProvider } = require(&#39;@opentelemetry/sdk-trace-node&#39;);<br>const { ConsoleSpanExporter, SimpleSpanProcessor } = require(&#39;@opentelemetry/sdk-trace-base&#39;);<br>const { JaegerExporter } = require(&#39;@opentelemetry/exporter-jaeger&#39;);\<br><br>const provider = new NodeTracerProvider();<br>// Export to Jaeger<br>const exporter = new JaegerExporter({<br>  endpoint: &#39;http://jaeger:14268/api/traces&#39;,<br>});<br>// Add processors<br>provider.addSpanProcessor(new SimpleSpanProcessor(exporter));<br>provider.addSpanProcessor(new SimpleSpanProcessor(new ConsoleSpanExporter()));<br>provider.register();</pre><p>✅ Now every request through your service gets traced.</p><h3>Step 2: Add Context Propagation 🧵</h3><p>Traces are only useful if they flow <strong>across services</strong>.<br> OpenTelemetry handles this with <strong>context propagation</strong>:</p><ul><li>Service A (frontend) starts a trace.</li><li>Service B (backend) continues the same trace with the same TraceID.</li></ul><p>No more broken trace chains!</p><h3>Step 3: Use the OpenTelemetry Collector</h3><p>Instead of sending data directly to Jaeger/Prometheus/Loki, use the <strong>Collector</strong>.</p><p>Why?</p><ul><li>Acts as a pipeline: receive → process → export.</li><li>Reduces vendor lock-in.</li><li>Centralizes configuration.</li></ul><p>A simple Collector config (exporting traces to Jaeger + metrics to Prometheus):</p><pre>receivers:<br>  otlp:<br>    protocols:<br>      http:<br>      grpc:<br>exporters:<br>  jaeger:<br>    endpoint: jaeger:14250<br>    tls:<br>      insecure: true<br>  prometheus:<br>    endpoint: &quot;0.0.0.0:9464&quot;<br>service:<br>  pipelines:<br>    traces:<br>      receivers: [otlp]<br>      exporters: [jaeger]<br>    metrics:<br>      receivers: [otlp]<br>      exporters: [prometheus]</pre><h3>Debugging in Action 🔍</h3><p>With OTel in place, you get:</p><ul><li><strong>Traces in Jaeger</strong> → see request paths across services.</li><li><strong>Metrics in Prometheus</strong> → monitor request rates, latencies, errors.</li><li><strong>Logs in Loki (optional)</strong> → correlate logs with trace IDs.</li></ul><p>Your stack suddenly goes from fragmented tools → to <strong>one observability story</strong>.</p><h3>Fun Challenges for You 🎮</h3><ul><li>Add <strong>custom spans</strong> for key business functions (e.g., “Checkout Service”).</li><li>Export both <strong>traces + metrics</strong> from your app and build a Grafana dashboard.</li><li>Pass a trace ID in logs → click from a log → jump straight to its trace!</li></ul><h3>Tips for Mastering OpenTelemetry 🧑‍💻</h3><ul><li>Start small → trace one service, then expand.</li><li>Always use the <strong>Collector</strong> (future-proofing).</li><li>Enrich spans with <strong>tags</strong> (userID, orderID, region).</li><li>Pair with Jaeger, Prometheus, Loki → for the holy trinity of observability.</li></ul><h3>What’s Next? 🌟</h3><p>Now that you’ve mastered the basics of OpenTelemetry, the next step is <strong>correlation</strong>: linking traces, logs, and metrics together. In the next blog, we’ll explore <strong>“Debugging Like a Detective: Correlating Jaeger Traces with Loki Logs”</strong> 🔎</p><p>Until then, happy instrumenting! 🚀</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=903a08960fc1" width="1" height="1" alt=""><hr><p><a href="https://blog.devops.dev/mastering-opentelemetry-instrumenting-your-microservices-like-a-pro-%EF%B8%8F-903a08960fc1">Mastering OpenTelemetry: Instrumenting Your Microservices Like a Pro 🛠️</a> was originally published in <a href="https://blog.devops.dev/">DevOps.dev</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Beyond Jaeger: Building a Unified Observability Stack with Traces, Logs, and Metrics ]]></title>
            <link>https://towardsdev.com/beyond-jaeger-building-a-unified-observability-stack-with-traces-logs-and-metrics-9430acb76572?source=rss-9697c9addf03------2</link>
            <guid isPermaLink="false">https://medium.com/p/9430acb76572</guid>
            <category><![CDATA[jaeger]]></category>
            <category><![CDATA[tracing]]></category>
            <dc:creator><![CDATA[NIKITA VERMA ]]></dc:creator>
            <pubDate>Sat, 14 Jun 2025 06:31:05 GMT</pubDate>
            <atom:updated>2025-06-26T14:35:52.253Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*OGhquMTsRFOgWflu" /></figure><p>Microservices are powerful — but debugging them can be a nightmare when you’re flying blind. Last time, we explored how Jaeger brings distributed tracing to your Kubernetes cluster, making it easier to see how requests flow across your services. But what if you could go even further?</p><p>Imagine diagnosing issues not just with traces, but by instantly jumping to related logs and metrics — all in one place. That’s the magic of a unified observability stack, and today, I’ll show you how to get there.</p><h3>Why Go Beyond Tracing?</h3><p>Distributed tracing is like the X-ray of your system, but real-world debugging often needs more:</p><ul><li>Correlate Traces with Logs: See the exact log lines for a failed or slow span.</li><li>Metrics for Context: Spot spikes in latency or error rates, then dig into traces and logs for the “why.”</li><li>End-to-End Visibility: One dashboard, all your answers.</li></ul><h3>The Stack: Jaeger, Loki, Prometheus, and Grafana</h3><p>Here’s the dream team:</p><ul><li>Jaeger: Distributed tracing</li><li>Loki: Aggregated, searchable logs</li><li>Prometheus: Powerful metrics collection</li><li>Grafana: Visualization and correlation for all three</li></ul><h3>Step-by-Step: Building Your Observability Playground</h3><h3>1️⃣ Deploy the Stack with Helm</h3><p>Assuming you have a Kubernetes cluster and Helm installed:</p><pre># Add Helm repos<br>helm repo add jaegertracing https://jaegertracing.github.io/helm-charts<br>helm repo add grafana https://grafana.github.io/helm-charts<br>helm repo update<br># Deploy Jaeger (tracing)<br>helm install jaeger jaegertracing/jaeger --namespace tracing --create-namespace<br># Deploy Loki (logging)<br>helm install loki grafana/loki-stack --namespace logging --create-namespace<br># Deploy Prometheus (metrics)<br>helm install prometheus prometheus-community/prometheus --namespace monitoring --create-namespace<br># Deploy Grafana (visualization)<br>helm install grafana grafana/grafana --namespace monitoring</pre><h3>2️⃣ Instrument Your App with OpenTelemetry</h3><p>OpenTelemetry is the new standard for sending traces, logs, and metrics. Here’s a Node.js example for tracing to Jaeger:</p><pre>const { NodeTracerProvider } = require(&#39;@opentelemetry/sdk-trace-node&#39;);<br>const { JaegerExporter } = require(&#39;@opentelemetry/exporter-jaeger&#39;);<br>const provider = new NodeTracerProvider();<br>const exporter = new JaegerExporter({ endpoint: &#39;http://jaeger:14268/api/traces&#39; });<br>provider.addSpanProcessor(new SimpleSpanProcessor(exporter));<br>provider.register();</pre><p>For logs, use a Loki-compatible logger (like <a href="https://github.com/JaniAnttonen/winston-loki">winston-loki</a>). For metrics, use the OpenTelemetry metrics SDK.</p><h3>3️⃣ Connect Everything in Grafana</h3><ul><li>Port-forward Grafana:</li></ul><pre>kubectl port-forward svc/grafana 3000:80 -n monitoring</pre><ul><li>Login at <a href="http://localhost:3000/">http://localhost:3000</a> (default user/pass: admin/admin).</li><li>Add data sources: Jaeger, Loki, and Prometheus.</li><li>Build dashboards: Correlate traces, logs, and metrics with a single click.</li></ul><h3>Hands-On Challenge: Debug Like a Pro 🕵️‍♀️</h3><ol><li>Break Something: Simulate a slow or failing service in your app.</li><li>Trace the Request: Use Jaeger to find the slow span.</li><li>Jump to Logs: In Grafana, click straight from the trace to the related logs in Loki.</li><li>Check Metrics: See if Prometheus shows a spike in errors or latency at the same time.</li></ol><p>You’ve just gone from “something’s wrong” to “here’s exactly what happened” — in minutes.</p><h3>Tips for Mastering Unified Observability</h3><ul><li>Sampling: Don’t overwhelm your stack — trace a percentage of requests.</li><li>Enrich Spans: Add tags (user ID, request type) for faster filtering.</li><li>Alerting: Set up alerts in Grafana for anomalies in metrics or trace errors.</li><li>Explore Tempo: For large-scale tracing, check out Grafana Tempo as a Jaeger alternative.</li></ul><h3>What’s Next? 🚀</h3><p>You’ve now got a full observability cockpit — traces, logs, and metrics, all working together. Next up: we’ll dive deeper into OpenTelemetry, explore advanced tracing backends like Tempo, and see how AI is changing the game for troubleshooting.</p><p>Stay tuned, and happy debugging! 🔎</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9430acb76572" width="1" height="1" alt=""><hr><p><a href="https://towardsdev.com/beyond-jaeger-building-a-unified-observability-stack-with-traces-logs-and-metrics-9430acb76572">Beyond Jaeger: Building a Unified Observability Stack with Traces, Logs, and Metrics 🚦</a> was originally published in <a href="https://towardsdev.com/">Towards Dev</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Distributed Tracing with Jaeger: Debugging Microservices Like a Pro ]]></title>
            <link>https://blog.devops.dev/distributed-tracing-with-jaeger-debugging-microservices-like-a-pro-083bed10c8b1?source=rss-9697c9addf03------2</link>
            <guid isPermaLink="false">https://medium.com/p/083bed10c8b1</guid>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[tracing]]></category>
            <category><![CDATA[cncf]]></category>
            <category><![CDATA[jaeger]]></category>
            <category><![CDATA[open-source]]></category>
            <dc:creator><![CDATA[NIKITA VERMA ]]></dc:creator>
            <pubDate>Sun, 23 Mar 2025 15:35:56 GMT</pubDate>
            <atom:updated>2025-03-24T18:55:07.670Z</atom:updated>
            <content:encoded><![CDATA[<p>Microservices are awesome — until something goes wrong. With requests bouncing between dozens of services, finding the culprit behind a slow response or a failed transaction can feel like searching for a needle in a haystack. That’s where <strong>distributed tracing</strong> comes in, and Jaeger is here to save the day! 🌟</p><h4>Why Distributed Tracing Matters</h4><p>Think of distributed tracing as a GPS for your application’s requests. It tracks a request’s journey across services, giving you:</p><ul><li><strong>Pinpoint Accuracy</strong>: Identify where latency or errors creep in.</li><li><strong>End-to-End Visibility</strong>: See the full path of a request, not just isolated logs.</li><li><strong>Performance Optimization</strong>: Spot bottlenecks and optimize accordingly.</li></ul><p>If logs are the heartbeat of your app, tracing is the X-ray — showing you the hidden connections.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*pxiieGubzraqozK5l8G1Eg.png" /></figure><h4>Enter Jaeger: The Open-Source Tracing Hero 🦸‍♂️</h4><p>Jaeger, built by Uber and now part of the CNCF, is a lightweight, powerful tool for distributed tracing. It’s designed to handle the chaos of microservices with:</p><ul><li><strong>Scalability</strong>: Works seamlessly in Kubernetes and beyond.</li><li><strong>Ease of Use</strong>: Integrates with popular frameworks like OpenTelemetry.</li><li><strong>Rich UI</strong>: Visualize traces with timelines and dependency graphs.</li></ul><p>Jaeger doesn’t just track requests — it tells the story of how they unfold.</p><h4>Setting Up Jaeger in Kubernetes 🚀</h4><p>Ready to get tracing? Here’s how to deploy Jaeger in your cluster.</p><p>Prerequisites</p><ul><li>A running Kubernetes cluster.</li><li>Helm installed (because who doesn’t love Helm?).</li></ul><p>Step 1: Install Jaeger</p><p>1️⃣ Add the Jaeger Helm repository:</p><p>helm repo add jaegertracing https://jaegertracing.github.io/helm-charts helm repo update</p><p>2️⃣ Install Jaeger with its all-in-one deployment (perfect for testing):</p><p>helm install jaeger jaegertracing/jaeger --namespace tracing --create-namespace</p><p>Step 2: Instrument Your Application</p><p>Jaeger needs your app to send traces. Use OpenTelemetry (the industry standard) to instrument your code:</p><ul><li>Add the OpenTelemetry SDK to your app (e.g., Python, Go, Node.js).</li><li>Configure it to export traces to Jaeger’s endpoint (http://jaeger:14268/api/traces).</li></ul><p>Example for a Node.js app:</p><p>const { NodeTracerProvider } = require(&#39;@opentelemetry/sdk-trace-node&#39;); const { JaegerExporter } = require(&#39;@opentelemetry/exporter-jaeger&#39;); const provider = new NodeTracerProvider(); const exporter = new JaegerExporter({ endpoint: &#39;http://jaeger:14268/api/traces&#39; }); provider.register();</p><p>Step 3: Access the Jaeger UI</p><p>1️⃣ Port-forward to Jaeger’s query service:</p><p>kubectl port-forward svc/jaeger-query 16686:16686 -n tracing</p><p>2️⃣ Open <a href="http://localhost:16686/">http://localhost:16686</a> in your browser and explore your traces!</p><h4>Jaeger in Action: What You’ll See 📈</h4><ul><li><strong>Timeline View</strong>: A waterfall of spans showing how long each service took.</li><li><strong>Dependency Graph</strong>: Visualize which services talk to each other.</li><li><strong>Error Tracking</strong>: Spot failed requests and drill into the details.</li></ul><p>It’s like a detective’s corkboard, but for your microservices.</p><h4>Fun Challenges for You! 🎮</h4><p>Test your tracing skills:</p><ul><li>Create a custom span to track a specific function in your app.</li><li>Simulate a slow service and use Jaeger to find the bottleneck.</li><li>Compare Jaeger’s dependency graph with your app’s architecture diagram — do they match?</li></ul><h4>Tips for Mastering Jaeger 🛠</h4><ul><li><strong>Use Sampling</strong>: Avoid overwhelming Jaeger by sampling traces (e.g., 1% of requests).</li><li><strong>Add Context</strong>: Enrich spans with tags like user IDs or request payloads.</li><li><strong>Pair with Loki</strong>: Combine tracing with logs for the ultimate debugging combo.</li></ul><h4>What’s Next? 🚀</h4><p>Jaeger turns microservices chaos into clarity. It’s the perfect companion to your Loki-Grafana setup, giving you logs <em>and</em> traces in harmony. Next time, we’ll explore how to tie it all together with OpenTelemetry — the glue that unifies observability. Until then, happy tracing! 🔎</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=083bed10c8b1" width="1" height="1" alt=""><hr><p><a href="https://blog.devops.dev/distributed-tracing-with-jaeger-debugging-microservices-like-a-pro-083bed10c8b1">Distributed Tracing with Jaeger: Debugging Microservices Like a Pro 🔍</a> was originally published in <a href="https://blog.devops.dev/">DevOps.dev</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Parsing a Tor Descriptor in Rust]]></title>
            <link>https://medium.com/@nikitackb456/parsing-a-tor-descriptor-in-rust-376c6634ce8e?source=rss-9697c9addf03------2</link>
            <guid isPermaLink="false">https://medium.com/p/376c6634ce8e</guid>
            <category><![CDATA[tor-project-open-source]]></category>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[rust]]></category>
            <category><![CDATA[tor-project]]></category>
            <dc:creator><![CDATA[NIKITA VERMA ]]></dc:creator>
            <pubDate>Sun, 23 Mar 2025 15:17:03 GMT</pubDate>
            <atom:updated>2025-03-23T15:17:03.629Z</atom:updated>
            <content:encoded><![CDATA[<p>My Understanding of the Task</p><p>The Tor Metrics Library (`metrics-lib`) is a Java tool that processes Tor network documents (descriptors) for analysis. For GSoC, we’re rewriting it in Rust. Issue #2 asks me to parse a simple descriptor and optionally save it to a database. I started with parsing because it’s the core step, and I wanted to nail it first. A sample Tor server descriptor looks like this:</p><p>// Simple Tor descriptor parser — by Nikita Verma</p><p>fn main() {</p><p>// Hardcoded Tor descriptor (a sample server descriptor)</p><p>let descriptor = “</p><p>router cool-router 10.0.0.1 9001 0 9030</p><p>published 2025–03–23 14:00:00</p><p>bandwidth 5000 10000 7500</p><p>“;</p><p>// Call the parser and handle the result</p><p>match parse_descriptor(descriptor) {</p><p>Ok(parsed) =&gt; {</p><p>println!(“🎉 Successfully parsed the descriptor!”);</p><p>println!(“🌐 Router: {} @ {}”, parsed.router_name, parsed.ip);</p><p>println!(“🚪 Port: {}”, parsed.or_port);</p><p>println!(“🕒 Published: {}”, parsed.published);</p><p>println!(“📈 Bandwidth: {} KB/s”, parsed.bandwidth_avg);</p><p>}</p><p>Err(e) =&gt; println!(“Oops! Something went wrong: {}”, e),</p><p>}</p><p>}</p><p>// Struct to store the parsed data</p><p>struct ServerDescriptor {</p><p>router_name: String,</p><p>ip: String,</p><p>or_port: u16, // Port numbers are small, so u16 is enough</p><p>published: String,</p><p>bandwidth_avg: u32, // Bandwidth in KB/s</p><p>}</p><p>// Function to parse the descriptor</p><p>fn parse_descriptor(data: &amp;str) -&gt; Result&lt;ServerDescriptor, String&gt; {</p><p>// Split the text into lines and remove empty ones</p><p>let lines: Vec&lt;&amp;str&gt; = data.trim().lines().collect();</p><p>// Check if we have at least 3 lines</p><p>if lines.len() &lt; 3 {</p><p>return Err(“Not enough lines in the descriptor!”.to_string());</p><p>}</p><p>// Parse the “router” line</p><p>let router_parts: Vec&lt;&amp;str&gt; = lines[0].split_whitespace().collect();</p><p>if router_parts.len() &lt; 6 || router_parts[0] != “router” {</p><p>return Err(“Router line is invalid!”.to_string());</p><p>}</p><p>// Parse the “published” line</p><p>let published_parts: Vec&lt;&amp;str&gt; = lines[1].split_whitespace().collect();</p><p>if published_parts.len() &lt; 3 || published_parts[0] != “published” {</p><p>return Err(“Published line is invalid!”.to_string());</p><p>}</p><p>// Parse the “bandwidth” line</p><p>let bandwidth_parts: Vec&lt;&amp;str&gt; = lines[2].split_whitespace().collect();</p><p>if bandwidth_parts.len() &lt; 4 || bandwidth_parts[0] != “bandwidth” {</p><p>return Err(“Bandwidth line is invalid!”.to_string());</p><p>}</p><p>// Build the struct with the parsed data</p><p>Ok(ServerDescriptor {</p><p>router_name: router_parts[1].to_string(),</p><p>ip: router_parts[2].to_string(),</p><p>or_port: router_parts[3].parse().map_err(|_| “Bad OR port!”.to_string())?,</p><p>published: format!(“{} {}”, published_parts[1], published_parts[2]),</p><p>bandwidth_avg: bandwidth_parts[1].parse().map_err(|_| “Bad bandwidth!”.to_string())?,</p><p>})</p><p>}</p><p>🎉 Successfully parsed the descriptor!</p><p>🌐 Router: cool-router @ 10.0.0.1</p><p>🚪 Port: 9001</p><p>🕒 Published: 2025–03–23 14:00:00</p><p>📈 Bandwidth: 5000 KB/s</p><h3>What I Learned</h3><ul><li>Rust basics: Structs, error handling with Result, and string parsing.</li><li>Tor descriptors: They’re structured text files with specific fields.</li><li>Keeping it simple: I focused on parsing first and added fun emojis to make the output pop!</li></ul><p>Next steps: I could add more fields (like dir_port or bandwidth_burst), read from a file, or save to a database like SQLite. I’ll explore those once I set up Rust locally or use Gitpod.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=376c6634ce8e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Exploring Proxyless Service Mesh with Dubbo: Unlocking the Future of Microservices Traffic…]]></title>
            <link>https://medium.com/@nikitackb456/exploring-proxyless-service-mesh-with-dubbo-unlocking-the-future-of-microservices-traffic-79ea1ae049d4?source=rss-9697c9addf03------2</link>
            <guid isPermaLink="false">https://medium.com/p/79ea1ae049d4</guid>
            <category><![CDATA[microservices]]></category>
            <category><![CDATA[dubbo]]></category>
            <category><![CDATA[apache]]></category>
            <category><![CDATA[proxyless]]></category>
            <category><![CDATA[open-source]]></category>
            <dc:creator><![CDATA[NIKITA VERMA ]]></dc:creator>
            <pubDate>Sun, 23 Mar 2025 07:46:31 GMT</pubDate>
            <atom:updated>2025-03-23T07:46:31.557Z</atom:updated>
            <content:encoded><![CDATA[<h3>Exploring Proxyless Service Mesh with Dubbo: Unlocking the Future of Microservices Traffic Management</h3><p>🚀 <strong>Have you ever wondered how microservices can communicate efficiently without the overhead of sidecar proxies?</strong> As I dive deeper into the world of service meshes, I’m discovering the exciting innovations happening around <strong>proxyless service mesh</strong> — a concept that eliminates unnecessary latency and simplifies architecture. In this blog, I’ll share my research and insights into how <strong>Apache Dubbo</strong> is leveraging proxyless service mesh to revolutionize microservices traffic management. Join me on this journey, and let’s explore together! 💡</p><h3>🌍 A New Contributor’s Perspective</h3><p>As a newcomer to the <strong>Apache Dubbo</strong> community, I am fascinated by how this project is evolving and pushing the boundaries of microservices architecture. My journey into <strong>proxyless service mesh</strong> has been full of learning, and I wanted to share these insights with fellow contributors and developers. This blog is not just a one-way conversation — it’s an open discussion! If you’re also exploring Dubbo, I’d love to hear your thoughts, feedback, and ideas. Let’s collaborate and build on each other’s knowledge to enhance our understanding and contributions! 🚀</p><h3>🔍 Why Proxyless Service Mesh?</h3><p>Traditional service meshes like Istio use <strong>sidecar proxies</strong> (e.g., Envoy) to manage traffic between services. While this ensures robust communication, it introduces <strong>latency, resource overhead, and deployment complexity</strong>. Enter <strong>proxyless service mesh</strong> — where services communicate <strong>directly with the control plane</strong>, reducing latency and simplifying operations. This means: ✅ Faster response times 🔥<br>✅ Less resource consumption 🚀<br>✅ Easier deployment &amp; maintenance ⚙️</p><p>Sounds cool, right? Let’s see how <strong>Apache Dubbo</strong> is embracing this shift!</p><h3>🌟 Apache Dubbo: Redefining Service Mesh with Proxyless Architecture</h3><p>Apache Dubbo, a leading RPC framework, has taken a <strong>bold step</strong> toward <strong>proxyless service mesh</strong>, allowing services to <strong>bypass sidecars</strong> while still managing traffic dynamically. This makes it an excellent choice for teams looking to optimize their <strong>microservices architecture</strong>.</p><p>But wait — how does it actually work? Let’s break it down! ⬇️</p><h3>🛠️ Key Traffic Management Features in Dubbo’s Proxyless Mesh</h3><p>Apache Dubbo’s proxyless service mesh offers <strong>some powerful traffic management capabilities</strong>, inspired by Istio’s approach. Here’s what makes it stand out:</p><h3>1️⃣ Dynamic Service Discovery (No More Sidecars!)</h3><p>Forget manual service registrations! Dubbo enables <strong>automatic service discovery</strong>, meaning services can find each other <strong>dynamically</strong> without needing sidecar proxies. This ensures smooth communication and minimizes downtime.</p><h3>2️⃣ Smart Traffic Routing (A/B Testing &amp; Canary Releases)</h3><p>Dubbo’s <strong>proportional traffic routing</strong> lets you experiment safely. Imagine you’re rolling out a new feature: 🟢 Route <strong>20% of users</strong> to the new version for testing.<br>🔵 Keep <strong>80% of users</strong> on the stable version.<br>🔄 Gradually shift traffic until the rollout is complete!</p><p>With <strong>seamless rollouts and zero-downtime deployments</strong>, Dubbo makes managing microservices <strong>a breeze</strong>! 🌬️</p><h3>3️⃣ Fine-Grained Mesh Routing Rules</h3><p>Inspired by <strong>Istio’s VirtualService and DestinationRule</strong>, Dubbo enables advanced <strong>traffic control</strong> based on service versions, labels, or request attributes. Need precise routing? Dubbo’s got you covered! 💪</p><h3>🚀 What’s Next? Expanding Dubbo’s Traffic Management Capabilities</h3><p>While Dubbo’s proxyless service mesh is already impressive, <strong>there’s so much more potential to unlock!</strong> Here are some exciting areas where we can push the boundaries:</p><h3>🔹 1. Strengthening xDS Protocol Integration</h3><p>To make Dubbo’s traffic management even more dynamic, we can: ✔️ Enhance <strong>real-time service registration</strong> 🏗️<br>✔️ Improve <strong>traffic rule updates on the fly</strong> 🔄<br>✔️ Enable <strong>instant policy enforcement</strong> for optimal performance ✅</p><h3>🔹 2. Supporting Advanced Traffic Scenarios</h3><p>What if we could add: 🎯 <strong>Weighted traffic splitting</strong> for smoother feature rollouts.<br>🛑 <strong>Circuit breaking</strong> to prevent cascading failures.<br>🔄 <strong>Request mirroring</strong> for safer debugging &amp; testing.</p><h3>🔹 3. Enhancing Security &amp; Observability</h3><p>A great service mesh must be <strong>secure and transparent</strong>! Let’s integrate: 🔐 <strong>Mutual TLS (mTLS)</strong> for encrypted communication.<br>📊 <strong>Observability tools</strong> like Prometheus &amp; Grafana for real-time monitoring.</p><p>By implementing these, Dubbo can <strong>compete with top-tier service meshes like Istio</strong> while keeping the architecture <strong>lightweight and efficient</strong>. 💡</p><h3>🔥 Real-World Example: Deploying a New Microservice with Dubbo</h3><p>Let’s say your team is launching <strong>a new version of a critical service</strong>. Here’s how Dubbo’s proxyless service mesh can make it stress-free: 1️⃣ <strong>Direct 20% of traffic</strong> to the new version for testing.<br>2️⃣ <strong>Monitor performance &amp; user feedback</strong> in real time.<br>3️⃣ <strong>Gradually shift 100% of traffic</strong> when confident.<br>4️⃣ 🎉 Enjoy a <strong>seamless, zero-downtime rollout</strong>!</p><p>Sounds like magic, right? ✨ But it’s <strong>real</strong> — and Dubbo makes it possible! 🚀</p><h3>🌎 Join the Conversation!</h3><p>The service mesh ecosystem is evolving fast, and <strong>proxyless architecture is the future!</strong> As a <strong>new contributor</strong>, I’m eager to continue learning and collaborating with the community. Whether you’re an experienced developer or just starting out like me, let’s discuss <strong>how we can improve and expand Dubbo’s proxyless capabilities together</strong>!</p><p>💬 <strong>What do you think about proxyless service mesh?</strong><br>🔧 <strong>Have you tried Apache Dubbo in your projects?</strong><br>📌 <strong>What features would you love to see next?</strong></p><p>Drop a comment below or reach out — I’d love to discuss and collaborate! 👇🔥</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=79ea1ae049d4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Unlocking Lightweight Kubernetes Logging with Loki and Grafana ]]></title>
            <link>https://blog.devops.dev/unlocking-lightweight-kubernetes-logging-with-loki-and-grafana-84814c614445?source=rss-9697c9addf03------2</link>
            <guid isPermaLink="false">https://medium.com/p/84814c614445</guid>
            <category><![CDATA[grafana]]></category>
            <category><![CDATA[cncf]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[logging]]></category>
            <category><![CDATA[loki]]></category>
            <dc:creator><![CDATA[NIKITA VERMA ]]></dc:creator>
            <pubDate>Tue, 14 Jan 2025 12:45:39 GMT</pubDate>
            <atom:updated>2025-01-16T17:24:07.615Z</atom:updated>
            <content:encoded><![CDATA[<p>Welcome back, Kubernetes adventurers! 🐳 After our logging escapades with Fluentd and Elasticsearch, it’s time to meet a lighter, faster duo that’s quickly gaining traction: <strong>Loki</strong> and <strong>Grafana</strong>. Think of Loki as the minimalist sibling of Elasticsearch, designed to do more with less — without compromising on power or efficiency.</p><p>In this blog, we’ll dive into the <strong>Loki-Grafana combo</strong> and discover why it’s the go-to solution for lightweight logging in Kubernetes.</p><h3>Why Should You Care About Logging? 🤔</h3><p>Logs are like the heartbeat of your applications. They provide crucial insights into what’s happening under the hood:</p><ul><li><strong>Debugging Made Easy</strong>: Spot errors and anomalies quickly.</li><li><strong>Performance Insights</strong>: Track trends and ensure optimal resource usage.</li><li><strong>Peace of Mind</strong>: Know what’s going on, even when you’re not looking.</li></ul><h3>Enter Loki: The Lightweight Logger 🌟</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*N9KLpsxdBbFIdWfJ" /></figure><p>Unlike Elasticsearch, Loki doesn’t index the contents of logs. Instead, it indexes only the <strong>metadata</strong> (like pod names, namespaces, etc.), making it:</p><ul><li><strong>Efficient</strong>: Uses less CPU and memory.</li><li><strong>Cost-Effective</strong>: Reduces storage requirements.</li><li><strong>Easy to Use</strong>: Integrates seamlessly with Grafana.</li></ul><p>Think of Loki as your logs’ librarian — it doesn’t read every page of a book but knows exactly where to find it.</p><h3>Meet Grafana: Loki’s Best Friend 📊</h3><p>Grafana is already a favorite for visualizing metrics, but paired with Loki, it becomes a powerful logging tool. Features include:</p><ul><li><strong>Dynamic Dashboards</strong>: Visualize logs alongside metrics in one interface.</li><li><strong>Advanced Querying</strong>: Use LogQL, Loki’s query language, for detailed insights.</li><li><strong>Alerts on Logs</strong>: Get notified about critical issues without extra effort.</li></ul><h3>How Loki Stacks Up Against Elasticsearch 🔄</h3><p><strong>FeatureLokiElasticsearchResource Usage</strong>MinimalHigh<strong>Setup Complexity</strong>SimpleComplex<strong>Search Mechanism</strong>Metadata-basedFull-text<strong>Cost</strong>LowHigh</p><p>Loki excels when you want <strong>lightweight and efficient logging</strong>, especially for Kubernetes workloads.</p><h3>Setting Up Loki and Grafana for Kubernetes 🚀</h3><h4>Prerequisites</h4><ul><li>A running <strong>Kubernetes cluster</strong>.</li><li><strong>Helm</strong> installed for deploying Loki and Grafana.</li></ul><h4>Step 1: Install Loki and Promtail</h4><p>Promtail is Loki’s log collector, responsible for shipping logs to Loki.</p><p>1️⃣ Add the Loki Helm repository:</p><pre>helm repo add grafana https://grafana.github.io/helm-charts  <br>helm repo update</pre><p>2️⃣ Install Loki and Promtail:</p><pre>helm install loki grafana/loki-stack --namespace kube-logging --create-namespace</pre><h4>Step 2: Install Grafana</h4><p>1️⃣ Deploy Grafana:</p><pre>helm install grafana grafana/grafana --namespace kube-logging</pre><p>2️⃣ Access Grafana:</p><pre>kubectl port-forward svc/grafana 3000:80 -n kube-logging</pre><p>Login with the default credentials (admin/admin).</p><h4>Step 3: Connect Grafana to Loki</h4><p>1️⃣ Add Loki as a data source in Grafana:</p><ul><li>Navigate to <strong>Configuration &gt; Data Sources</strong>.</li><li>Select <strong>Loki</strong> and enter its endpoint (http://loki:3100).</li></ul><p>2️⃣ Import prebuilt dashboards or create your own for a tailored view of your logs.</p><h3>Fun Challenges for You! 🎮</h3><p>Ready to put your skills to the test?</p><ul><li>Create a <strong>custom LogQL query</strong> to filter error logs.</li><li>Build a <strong>Grafana alert</strong> for high-frequency errors in a specific namespace.</li><li>Compare the Loki-Grafana setup with EFK (Elasticsearch-Fluentd-Kibana) in terms of resource usage.</li></ul><h3>Tips for Mastering Loki and Grafana 🛠</h3><ul><li><strong>Structured Logs</strong>: Use JSON logs for better metadata parsing.</li><li><strong>Retention Policies</strong>: Configure retention to avoid excessive storage use.</li><li><strong>Explore Dashboards</strong>: Grafana has a rich library of templates to kickstart your setup.</li></ul><h3>What’s Next? 🚀</h3><p>Loki and Grafana prove that logging doesn’t have to be heavyweight to be effective. With their simplicity and power, they’re perfect for Kubernetes workloads, especially when efficiency matters.</p><p>Stay tuned for our next blog where we’ll dive into <strong>distributed tracing with Jaeger</strong> — the secret sauce for debugging microservices! Until then, happy logging! 📝</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=84814c614445" width="1" height="1" alt=""><hr><p><a href="https://blog.devops.dev/unlocking-lightweight-kubernetes-logging-with-loki-and-grafana-84814c614445">Unlocking Lightweight Kubernetes Logging with Loki and Grafana 🚀</a> was originally published in <a href="https://blog.devops.dev/">DevOps.dev</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Kubernetes Logging Adventures with Fluentd and Elasticsearch]]></title>
            <link>https://towardsdev.com/kubernetes-logging-adventures-with-fluentd-and-elasticsearch-54c36ed1b768?source=rss-9697c9addf03------2</link>
            <guid isPermaLink="false">https://medium.com/p/54c36ed1b768</guid>
            <category><![CDATA[fluentd]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[logging]]></category>
            <category><![CDATA[elasticsearch]]></category>
            <category><![CDATA[open-source]]></category>
            <dc:creator><![CDATA[NIKITA VERMA ]]></dc:creator>
            <pubDate>Fri, 10 Jan 2025 03:17:54 GMT</pubDate>
            <atom:updated>2025-01-10T15:18:18.349Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*yK-_A4rAKh40pZAz" /></figure><p><strong>Welcome back, Kubernetes explorers!</strong> After diving into observability with Prometheus and Grafana, it’s time to level up. Let’s talk about logging — the detective work of DevOps. Logs are like breadcrumbs left by your applications, and our mission is to gather them into a single, easy-to-analyze basket.</p><p>🎯 In this adventure, we’ll:</p><ol><li>Set up <strong>Fluentd</strong> to gather logs.</li><li>Use <strong>Elasticsearch</strong> to store and search through them.</li><li>Ensure you feel like a Kubernetes logging pro by the end!</li></ol><h3>Why Do Logs Matter Anyway? 🕵️‍♀️</h3><p>Logs are like diaries for your applications. They tell stories about errors, performance, and everything in between. Without centralized logging, you’d be digging through multiple nodes just to find a tiny log file. Nobody’s got time for that, right?</p><p>With centralized logging, you can:<br>✔️ Detect issues faster than a superhero 🦸‍♀️.<br>✔️ Analyze trends with ninja-level precision 🥷.<br>✔️ Sleep better at night knowing your logs are safe 😴.</p><h3>The Stars of Our Show 🌟</h3><p><strong>🛠 Fluentd</strong>: The log collector. Think of it as the log detective, sniffing out log files and sending them to Elasticsearch.<br><strong>📚 Elasticsearch</strong>: The search engine. It stores logs and helps you find that needle in the haystack with blazing speed.</p><h3>Mission: Centralized Logging Setup 🎯</h3><p>Here’s your step-by-step guide to Kubernetes logging glory. Ready? Let’s go!</p><h4>Step 1: Set Up Elasticsearch</h4><p>Elasticsearch is like the library where all your logs will be stored.</p><p>👩‍💻 <strong>Command Time!</strong></p><p>Add the Helm repo:</p><pre>helm repo add elastic https://helm.elastic.co   <br>helm repo update</pre><p>Install Elasticsearch:</p><pre>helm install elasticsearch elastic/elasticsearch --namespace kube-logging --create-namespace</pre><p><strong>📖 Fun fact</strong>: Did you know Elasticsearch was initially created for full-text search in websites?</p><h4>Step 2: Fluentd to the Rescue! 🦸‍♂️</h4><p>Fluentd is the Robin to Elasticsearch’s Batman.</p><p>👩‍💻 <strong>Time for Some YAML Magic!</strong></p><p>Create a Fluentd ConfigMap:</p><pre>apiVersion: v1   kind: ConfigMap   metadata:     name: fluentd-config     namespace: kube-logging   data:     fluent.conf: |       &lt;source&gt;         @type tail         path /var/log/containers/*.log         tag kubernetes.*       &lt;/source&gt;       &lt;match kubernetes.**&gt;         @type elasticsearch         host elasticsearch.kube-logging.svc.cluster.local       &lt;/match&gt;</pre><p>Deploy Fluentd as a DaemonSet:</p><pre>apiVersion: apps/v1   kind: DaemonSet   metadata:     name: fluentd     namespace: kube-logging   spec:     template:       spec:         containers:           - name: fluentd             image: fluent/fluentd:v1.14.2             volumeMounts:               - name: varlog                 mountPath: /var/log               - name: config-volume                 mountPath: /fluentd/etc</pre><p>🎉 Congrats! Fluentd is now collecting logs from all your pods and sending them to Elasticsearch.</p><h3>How Do You Know It’s Working? 🔍</h3><p>Let’s verify our setup:</p><p><strong>Check Pods</strong>:</p><pre>kubectl get pods -n kube-logging</pre><p>All pods should be running. If not, don’t panic! Troubleshooting is part of the fun.</p><p><strong>Explore Elasticsearch</strong>:</p><pre>kubectl port-forward svc/elasticsearch-master 9200:9200 -n kube-logging<br></pre><h3>Fun Challenges for You! 🎮</h3><ol><li>Create a custom Fluentd filter to exclude debug logs.</li><li>Build a Grafana dashboard to visualize error trends from your logs.</li><li>Bonus points if you deploy the EFK stack (Elasticsearch, Fluentd, Kibana) and share screenshots!</li></ol><h3>Tips from the Trenches 🛠</h3><p>🔹 <strong>Structured Logs Are Gold</strong>: Emit logs in JSON format for better parsing.<br>🔹 <strong>Rotate Logs</strong>: Prevent disk space nightmares with log rotation policies.<br>🔹 <strong>Mind Your Data</strong>: Avoid logging sensitive info.</p><h3>What’s Next? 🚀</h3><p>In our next blog, we’ll explore <strong>Kubernetes logging with Loki and Grafana</strong> — the dynamic duo of lightweight logging! Stay tuned, and as always, happy logging!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=54c36ed1b768" width="1" height="1" alt=""><hr><p><a href="https://towardsdev.com/kubernetes-logging-adventures-with-fluentd-and-elasticsearch-54c36ed1b768">Kubernetes Logging Adventures with Fluentd and Elasticsearch</a> was originally published in <a href="https://towardsdev.com/">Towards Dev</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[K8s Observability Prometheus]]></title>
            <link>https://blog.devops.dev/k8s-observability-prometheus-960c490e6981?source=rss-9697c9addf03------2</link>
            <guid isPermaLink="false">https://medium.com/p/960c490e6981</guid>
            <category><![CDATA[kubernetes-cluster]]></category>
            <category><![CDATA[prometheus]]></category>
            <category><![CDATA[observability]]></category>
            <category><![CDATA[cncf]]></category>
            <category><![CDATA[open-source]]></category>
            <dc:creator><![CDATA[NIKITA VERMA ]]></dc:creator>
            <pubDate>Tue, 07 Jan 2025 09:30:16 GMT</pubDate>
            <atom:updated>2025-01-09T15:41:09.588Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>Observability in Kubernetes with Prometheus and Grafana</strong></p><p><strong>Introduction</strong> Observability is a key aspect of managing Kubernetes applications. It helps you monitor performance, detect issues, and ensure applications are running smoothly. In this blog, we’ll focus on two popular tools — <strong>Prometheus</strong> and <strong>Grafana</strong> — that make observability simple and effective for Kubernetes beginners.</p><h3>Why Observability Matters in Kubernetes</h3><p>Kubernetes runs distributed systems, making it challenging to track resource usage and performance across multiple pods and nodes. Observability tools help by:</p><ul><li><strong>Collecting Metrics:</strong> Monitor CPU, memory, and network usage.</li><li><strong>Visualizing Data:</strong> Display trends and patterns using dashboards.</li><li><strong>Setting Alerts:</strong> Notify teams of potential issues before they escalate.</li></ul><h3>What is Prometheus?</h3><p>Prometheus is an open-source monitoring system that collects and stores metrics as time-series data. It is highly scalable and integrates seamlessly with Kubernetes.</p><h3>Key Features:</h3><ul><li><strong>Time-Series Data Storage:</strong> Stores data efficiently for historical analysis.</li><li><strong>Flexible Query Language (PromQL):</strong> Enables custom queries to analyze metrics.</li><li><strong>Built-in Alerting:</strong> Supports rule-based alerts to detect anomalies.</li></ul><h3>What is Grafana?</h3><p>Grafana is an open-source platform used to visualize and analyze metrics through dashboards. It integrates with Prometheus to provide intuitive charts and graphs.</p><h3>Key Features:</h3><ul><li><strong>Interactive Dashboards:</strong> Allows dynamic exploration of data.</li><li><strong>Multi-Source Support:</strong> Connects to multiple databases and tools.</li><li><strong>Custom Alerts and Notifications:</strong> Sends alerts via email, Slack, or other tools.</li></ul><h3>Setting Up Prometheus and Grafana in Kubernetes</h3><p>Let’s get started with a simple setup:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*dW9moEYXpIfXwJK3" /></figure><h3>Step 1: Install Prometheus and Grafana with Helm</h3><p>Helm simplifies Kubernetes deployments. Use the following commands:</p><pre>helm repo add prometheus-community https://prometheus-community.github.io/helm-charts<br>helm repo update</pre><pre>helm install prometheus prometheus-community/kube-prometheus-stack</pre><h3>Step 2: Access Grafana Dashboard</h3><pre>kubectl port-forward svc/prometheus-grafana 3000:80</pre><p>Open your browser and go to http://localhost:3000. Use the default credentials (admin/admin) to log in.</p><h3>Step 3: Import a Dashboard</h3><ul><li>Go to Grafana’s UI, navigate to <strong>Dashboards &gt; Import</strong>, and paste a Dashboard ID from Grafana’s library.</li><li>For example, use ID 3119 for Kubernetes cluster monitoring.</li></ul><h3>Step 4: Explore Metrics in Prometheus</h3><p>Access Prometheus by forwarding its port:</p><pre>kubectl port-forward svc/prometheus-kube-prometheus-prometheus 9090:9090</pre><p>Visit http://localhost:9090 to explore metrics like node_cpu_seconds_total.</p><h3>Common Metrics to Monitor</h3><ul><li><strong>CPU Usage:</strong> node_cpu_seconds_total</li><li><strong>Memory Usage:</strong> node_memory_MemAvailable_bytes</li><li><strong>Pod Restarts:</strong> kube_pod_container_status_restarts_total</li><li><strong>Network Traffic:</strong> node_network_receive_bytes_total</li></ul><h3>Creating Alerts in Prometheus</h3><p>Alerts can prevent critical failures by notifying you of potential issues. Here’s how to create a simple alert rule:</p><h3>Step 1: Create an Alert Rule File</h3><pre>group:<br>  - name: example.rules<br>    rules:<br>      - alert: HighCPUUsage<br>        expr: node_cpu_seconds_total &gt; 80<br>        for: 5m<br>        labels:<br>          severity: critical<br>        annotations:<br>          summary: &#39;High CPU usage detected&#39;</pre><h3>Step 2: Apply the Rule</h3><p>Save the rule file and update Prometheus configuration.</p><pre>kubectl apply -f alert-rule.yaml</pre><h3>Key Tips for Beginners</h3><p><strong>Start with Prebuilt Dashboards:</strong> Grafana has a rich library of templates to get you started quickly.</p><p><strong>Learn PromQL Basics:</strong> Focus on writing simple queries for monitoring resource usage.</p><p><strong>Automate Alerts:</strong> Use Prometheus alert rules to get notified of critical issues.</p><p><strong>Experiment and Explore:</strong> Don’t hesitate to test queries and settings in a local setup.</p><h3>Interactive Challenge for You!</h3><p>Ready to test your skills? Try the following mini-challenges:</p><ol><li>Create a custom alert in Prometheus to monitor memory usage exceeding 70% for 10 minutes.</li><li>Design your own Grafana dashboard to visualize pod restarts and network traffic trends.</li><li>Share your dashboard screenshots and queries with the community!</li></ol><h3>Conclusion</h3><p>Observability in Kubernetes doesn’t have to be overwhelming. By using Prometheus and Grafana, you can monitor your clusters, visualize performance, and set up alerts effortlessly. Start small, experiment, and gradually expand your monitoring setup as your applications scale.</p><p><strong>Coming Next:</strong> Stay tuned for our next blog post, where we’ll explore Kubernetes logging with Fluentd and Elasticsearch to take your observability to the next level!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=960c490e6981" width="1" height="1" alt=""><hr><p><a href="https://blog.devops.dev/k8s-observability-prometheus-960c490e6981">K8s Observability Prometheus</a> was originally published in <a href="https://blog.devops.dev/">DevOps.dev</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>