<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Emre Karadeniz on Medium]]></title>
        <description><![CDATA[Stories by Emre Karadeniz on Medium]]></description>
        <link>https://medium.com/@emre.karadeniz?source=rss-df6a539b109f------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*HdCkv406c7DF1sBOhYjOlA@2x.jpeg</url>
            <title>Stories by Emre Karadeniz on Medium</title>
            <link>https://medium.com/@emre.karadeniz?source=rss-df6a539b109f------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 05:39:52 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@emre.karadeniz/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[An easy way to build custom couchbase kafka connector plug-in]]></title>
            <link>https://medium.com/@emre.karadeniz/an-easy-way-to-build-custom-couchbase-kafka-connector-673277b2f23?source=rss-df6a539b109f------2</link>
            <guid isPermaLink="false">https://medium.com/p/673277b2f23</guid>
            <category><![CDATA[kafka]]></category>
            <category><![CDATA[connector]]></category>
            <category><![CDATA[couchbase]]></category>
            <dc:creator><![CDATA[Emre Karadeniz]]></dc:creator>
            <pubDate>Wed, 19 Apr 2023 08:24:25 GMT</pubDate>
            <atom:updated>2023-04-19T08:26:28.095Z</atom:updated>
            <content:encoded><![CDATA[<p>Couchbase is a powerful NoSQL database that can integrate with other tools through connectors. One such connector is the Kafka connector, which provides a number of useful features. Because it is an open source project, users can customize it to meet their specific needs.</p><p>Users can customize the Kafka connector in a variety of ways, such as by modifying its configuration settings or by developing custom serializers and deserializers to handle complex data types.</p><h3>Get source code from github</h3><p><a href="https://github.com/couchbase/kafka-connect-couchbase">GitHub - couchbase/kafka-connect-couchbase: Kafka Connect connector for Couchbase Server</a></p><h3>Customize the source code</h3><p>In this article, I demonstrate how to update the AllPassFilter class to add new functionality.</p><pre><br>package com.couchbase.connect.kafka.filter;<br><br>import com.couchbase.connect.kafka.handler.source.DocumentEvent;<br><br>public class AllPassFilter implements Filter {<br><br>  @Override<br>  public boolean pass(final DocumentEvent event) {<br>    return !event.key().startsWith(&quot;_txn:&quot;);<br>  }<br>}</pre><p>This functionality ignores events that have a key starting with _txn:.</p><p>This approach is useful in cases where certain events, such as the _txn:client-record document, should be excluded from processing. By implementing a filter that ignores events with a specific key prefix, it is possible to streamline event processing and ensure that only relevant events are processed.</p><blockquote>In my use case, Couchbase creates a document with the key _txn:client-record to keep track of transactions. This record is modified every time an upsert operation occurs on the bucket. My Couchbase Kafka connector was producing this record to a Kafka topic, which I wanted to avoid. To address this, I implemented a control mechanism.</blockquote><blockquote>Overall, the key _txn:client-record is a special document used by Couchbase to manage transactions. However, in some cases, such as when using the Couchbase Kafka connector, it may be desirable to exclude this document from being produced to a Kafka topic. By implementing a control mechanism, it is possible to selectively include or exclude this document based on specific needs and use cases.</blockquote><h3>Build and Dockerize connector</h3><p>To build the source code</p><pre>mvn clean package</pre><p>Use the following dockerfile to dockerize</p><pre>FROM FROM confluentinc/cp-kafka-connect:6.2.0<br><br>RUN confluent-hub install --no-prompt couchbase/kafka-connect-couchbase:4.1.4<br>COPY target/components/packages/couchbase-kafka-connect-couchbase-4.1.4/couchbase-kafka-connect-couchbase-4.1.4/lib/kafka-connect-couchbase-4.1.4.jar /usr/share/confluent-hub-components/couchbase-kafka-connect-couchbase</pre><p>Now, you have an image that includes both the Kafka Connect framework and your customized Couchbase Kafka connector plugin installed.</p><pre>INFO Poll returns 0 result(s) (filtered out 1) (com.couchbase.connect.kafka.CouchbaseSourceTask)</pre><p>Now, the connector is able to filter out documents that have the key _txn:client-record</p><p>I hope this helps you.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=673277b2f23" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Istio Request Control with Envoy Filters — Request Headers]]></title>
            <link>https://medium.com/hepsiburadatech/istio-request-control-with-envoy-filters-request-headers-43dbd0dba7a9?source=rss-df6a539b109f------2</link>
            <guid isPermaLink="false">https://medium.com/p/43dbd0dba7a9</guid>
            <category><![CDATA[istio]]></category>
            <category><![CDATA[envoy-filter]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[k8s]]></category>
            <category><![CDATA[devops]]></category>
            <dc:creator><![CDATA[Emre Karadeniz]]></dc:creator>
            <pubDate>Tue, 14 Jul 2020 19:22:49 GMT</pubDate>
            <atom:updated>2020-08-10T08:53:55.982Z</atom:updated>
            <content:encoded><![CDATA[<h3>Istio Request Control with Envoy Filters — Request Headers</h3><p>In microservices each service has it’s own task. To execute each task, microservices receive requests and each request may contain headers, body, or query strings. While changing any part of a request that you receive, you have to think twice. Because you may cause an exception on other microservices. Let say all of your applications expect a header parameter called user-id . What happens if the request sender change the name of this header? All of your application gets errors. In a same manner, let assume that you get JWT from Authorization header. If JWT payload changes, again your applications that uses JWT payload gets errors.</p><p>Let assume that your end-user login from user service and gets a JWT. On each request this JWT setted on Authorization header. Other microservices resolve this JWT and obtain payload to make some calculations.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/491/1*zC_pT3S_nKxvcji-WFXK2w.png" /><figcaption>Book service decode jwt from Authorization header and make some controls by using payload of JWT</figcaption></figure><p>Above graph is a simple example. Imagine you have lots of micro services like book service and each of them uses different technologies like .Net Core, Go, Java etc. For each service you have to create a JWT parsing module or library. When your teams get bigger, this become confused. Because JWTs may change over time. A claim can be removed or modified, or a new claim can be added for a purpose. Any change on JWT, forces you to make a development. As a result of this a deployment pipeline should be runned.</p><p>Istio can help us at this point. Istio describe itself like,</p><blockquote>“I make it easy to create a network of deployed services with load balancing, service-to-service authentication, monitoring, and more..”</blockquote><p>Simply when you install Istio to your kubernetes cluster and make one of your namespaces as istio injected, all request to workloads on this namespace pass through Istio architecture.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/769/1*LSPuvMzMDnCP9uoOCwM1LA.png" /><figcaption>Book service handles jwt</figcaption></figure><p>On the above everything looks fine. You have one service and handle JWT on this service.</p><p>As I said before, when your teams get bigger and new services appear, your graph looks like that.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/783/1*VnM_XgG5fuNbESTmfURXlA.png" /></figure><p>Parsing JWT on each service can be confused. When your user service changes JWT payload, you have to update your parsing code on each of your applications.</p><p>With Istio, JWT and other request headers can be controlled before the request hit to your services. There are a few ways to control your request headers on Istio. I will mention about them. By applying these solutions, you don’t need to parse JWT on your applications. Istio will parse JWT for all your applications. We will use Envoy Filters to do this. Following graph illustrate it. When a request hit to envoy, our EnvoyFilters will be applied, then the request will be distributed to workloads. We will remove parsing code from our applications.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/771/1*aX64Mr94mIVpB0RPgwp0CQ.png" /><figcaption>JWT parsing or other header params handled on Envoy with envoy filters. Application do not need to handle request headers or JWT anymore</figcaption></figure><p>In theory, we understand what we want to achieve. Lets make it real.</p><p>First I created a namespace called book</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/3b5415bd15cef0075271c4651d0b1348/href">https://medium.com/media/3b5415bd15cef0075271c4651d0b1348/href</a></iframe><p>Then I need an application that runs on book namespace. I just create a simple .net core application. It simply gets user-id and is-authenticatedparameters from request header and returns the list of books.</p><blockquote>Note that user-id and is-authenticated params used to be obtained from JWT payload. We will parse JWT before request received from the service and set them as header parameters. With this way, our applications will not be responsible for jwt parsing.</blockquote><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/39fb96039bf2e0faea990e32e23be041/href">https://medium.com/media/39fb96039bf2e0faea990e32e23be041/href</a></iframe><p>Let inject Istio to our namespace by labeling namespace. The following kubectl command labels book namesapce as istio injected.</p><pre>kubectl label namespace book istio-injection=enabled</pre><blockquote>Now, we need to create a gateway to receive requests from istio envoy proxy. Istio has a tutorial for that. You can read and learn more about <a href="https://istio.io/latest/docs/examples/bookinfo/"><strong>here</strong></a>.</blockquote><p>When you install Istio to your k8s cluster, it creates a namespace called istio-system. Istio creates a service called istio-ingressgateway . This is the default controller and entry point to our mesh. For now we are dealing with http request, we must find the http port of ingress gateway. You can find it by editing the service definition like following.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*f0gWX_gBIMtT4u-6cmtdzg.png" /><figcaption>http2 is the entry point for http request</figcaption></figure><p>As you can see Istio has port definitions for each connection type. We will use http2 for http request. As seen on the screenshot our target port must be 31380. We need to orient all of our dns’ to 31380 port of our k8s cluster. Istio will handle all network traffic for us.</p><p>We need to define a gateway to handle requests for our book service. Below configuration says that all external request that comes from http protocol, will be redirected to service which has a selector like istio: ingressgateway. This is default controller of Istio. Note that hosts parameter defined as * . This means that all request with any hosts will be handled by the ingressgateway controller.</p><blockquote>You can specify special domians like “*.myteam.mycompany.com” ashosts property.</blockquote><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/593420e24739a56c92fab687df664613/href">https://medium.com/media/593420e24739a56c92fab687df664613/href</a></iframe><p>Now, we forwarded incoming requests to istio ingress gateway. To distribute each request to related services we need to define VirtualServices. Let create a virtual service for our BookService .</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2313cb7aa0395a7fe77ca329f76e109a/href">https://medium.com/media/2313cb7aa0395a7fe77ca329f76e109a/href</a></iframe><p>This configuration implies that, if our ingress gateway receive a request that has host bookservice.example.com then the request will be routed to book-service-svc in the book namespace.</p><p>Our k8s sevices may have multiple subsets. Subsets can be considered as deployments or deamons sets. On the virtual service config all request forwarded to v01 subset.</p><p>When we create a deployment we can add labels to this deployments to distinguish from themselves. In istio we can distributed requests to specific deployment. To do this we need to create DestinationRules. For our example, lets create a destination rule for v01.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/05ea7cb87b6e2f8aaf6b1739f871e934/href">https://medium.com/media/05ea7cb87b6e2f8aaf6b1739f871e934/href</a></iframe><p>This configuration implies that, in book namespace, there exists a book-service-svc and it has a selector label to which deployments will be used. <br>This destination rule points the deployment which has a version as label and v01 as value. If you use v01 as subset in the VirtualService configuration then, the requests will be forwarded to Deployment which has version: v01 label. To clarify look at the graph.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/997/1*5S0Q07m0o9TWteNmEdZm0A.png" /><figcaption>Book Service has multiple version of deployments</figcaption></figure><p>Book service select v01 as workload. It is defined in VirtualService configuration. To select other subset as workload you need to define destination rules for each version and add them as subsets. A virtual service can have multiple subsets.</p><h3>Request Header Controlling</h3><p>For now, our book service works but any header parameters hasn’t been passed.</p><p>When we request books, it return nothing for now.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/575/1*dzia9Nw1Ay5xNLTdY14uqQ.png" /></figure><p>We will control requests with Istio and pass required headers to our application with the keys that is understandable by the application.</p><p>First, lets assume that front-end application knows the current user’s id and authentication state. While requesting for books front-end add header fields like customer-id=somecustomerid and logged-in=true . These fields unfamiliar to our microservices (Look at our .net code. We expect different header names). To align with front-end application, we need to update our code. Instead of updating our applications we will create an EnvoyFilter. You can read more about Istio’s EnvoyFilter <a href="https://istio.io/latest/docs/reference/config/networking/envoy-filter/"><strong>here</strong></a>.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7a203fba00fca85c2bac271606706035/href">https://medium.com/media/7a203fba00fca85c2bac271606706035/href</a></iframe><p>This configuration implies that, when book-service-svc on book namespace will receive a request, the headers of request will be modified. The lua script in the yaml, maps current customer-id and logged-in fields to user-id and is-authenticated fields. EnvoyFilter modifies the request header before request reaches to our services. With this way our applications will not effect from header parameter changes. Now if we call book service again, we get the result.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/758/1*TqLRFoG_oV_YsVnEzGkrcA.png" /></figure><p>Our application doesn’t know customer-id and logged-in fields. Istio EnvoyFilter will change customer-id and logged-in fields in a way that our application can understand.</p><p>Now, lets try something more complicated. This time we will try parsing JWT with EnvoyFilter. In microservices JWT is generally generated by the user service and passed to all other services with cookies or Authorization header. Sometimes you need to parse JWT to get some payload information to use on your applications like user name, id etc. When a claim’s name in JWT payload changes what happens? You need to update all of your applications. To avoid this situation, we may parse it with an EnvoyFilter and pass required payload fields to our applications.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/df150b81fd9720073daa46156f708d86/href">https://medium.com/media/df150b81fd9720073daa46156f708d86/href</a></iframe><p>This configuration may looks very complicated, but is a lua script that decodes JWT and then puts decoded claims into request as header parameters. This is useful when you need to know payload claims. I found this config from github. You can check the git <a href="https://github.com/BarDweller/istio-content-based-routing/blob/master/istio-envoy-filter-jwt-lua.yaml"><strong><em>repository</em></strong></a>. In the last part of lua script, I get the customer-id and logged-in fields from the decoded jwt and put them into request as header parameters.</p><blockquote>Our services do not need to parse JWT. Istio envoy filter can decode it for all of our services and serve the desired claims as header parameters. We don’t need a JWT parser library. If any of the claim changes, I only need the update Istio Envoy Filter configuration.</blockquote><p><strong>In conclusion, </strong>Istio can help us controlling our request headers. Instead of controlling request headers on your applications, you can centralize this control mechanism with Istio. This makes your architecture more flexible. Your services are not affected by the changes made by the other services. You just update your envoy filter.</p><h3>Notes</h3><p>The first approach is a good solution for request header controlling for your services. JWT approach isn’t an advanced solution. It only finds with simple claims, nested object or arrays won’t work. You may need to write better script for this. The developer of this lua script mentions more about on <a href="https://github.com/BarDweller/istio-content-based-routing">github</a>.</p><p>Envoy filters may cause some performance problems on your service mesh. Custom lua scripts should be tested before production.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=43dbd0dba7a9" width="1" height="1" alt=""><hr><p><a href="https://medium.com/hepsiburadatech/istio-request-control-with-envoy-filters-request-headers-43dbd0dba7a9">Istio Request Control with Envoy Filters — Request Headers</a> was originally published in <a href="https://medium.com/hepsiburadatech">hepsiburadatech</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Control your Linux servers by using a Go Api]]></title>
            <link>https://medium.com/hepsiburadatech/control-your-ssh-servers-by-using-a-go-api-779a68557826?source=rss-df6a539b109f------2</link>
            <guid isPermaLink="false">https://medium.com/p/779a68557826</guid>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[ssh]]></category>
            <category><![CDATA[kubectl]]></category>
            <category><![CDATA[kubernetes]]></category>
            <dc:creator><![CDATA[Emre Karadeniz]]></dc:creator>
            <pubDate>Fri, 25 Oct 2019 11:01:33 GMT</pubDate>
            <atom:updated>2019-10-29T15:51:23.848Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/404/1*QLTvd2004LhrhMq3b7Mv5w.jpeg" /></figure><p>As a software developer you can do many tasks other than writing codes like create a docker image, publish a new version of your app to Kubernetes cluster etc. While doing this kind of tasks we are running some commands over ssh servers. Go has an ability to run ssh commands for you. In this article i will talk about this feature of Go by showing examples of how to execute ssh commands on our Kubernetes clusters.</p><p><strong>First Part: Create an SSH Connection</strong></p><p>In this part, we will create an ssh connection by using the <strong>“golang.org/x/crypto/ssh”</strong> library. We will create an interface called ISshClient. By defining this interface, our ssh provider will be more readable, testable and injectable.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d480a13eafcefdb181c1afcc84da44d2/href">https://medium.com/media/d480a13eafcefdb181c1afcc84da44d2/href</a></iframe><p>Then we need an implemantation for this interface. “golang.org/x/crypto/ssh” provides multiple authentication methods. We will use Password authentication. You can learn more about <strong>AuthMethod</strong> from <a href="https://godoc.org/golang.org/x/crypto/ssh#AuthMethod">here</a>.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/823d7cb40ad1e003e464e47673004123/href">https://medium.com/media/823d7cb40ad1e003e464e47673004123/href</a></iframe><blockquote>Our config contains a property which is called HostKeyCallback. This property is used for verification of server keys. In this example we use InsecureIgnoreHostKey. That means the server accept any host key. Beware of not using this property in production. Learn more about <a href="https://godoc.org/golang.org/x/crypto/ssh#HostKeyCallback"><strong>HostKeyCallback</strong></a>.</blockquote><p><strong>Second Part: Run SSH Commands</strong></p><p>I created an interface called ISshCommandExecuter contains RunSshCommand method to run ssh commands.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e0a3c1b0a183351cc9c49ad3e4fd5b66/href">https://medium.com/media/e0a3c1b0a183351cc9c49ad3e4fd5b66/href</a></iframe><p>To implement this interface we need to inject our ISshClient because we’ll get the connection from our ssh client then create a new session. When we create a new session, a connection establishes between the given remote machine and our go app. After that, you can run any command on the remote machine just like you’re using PuTTY.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c437b31ce7243e9b1a8260e8e61af942/href">https://medium.com/media/c437b31ce7243e9b1a8260e8e61af942/href</a></iframe><p><strong>What’s next?</strong></p><p>Now we are able to connect any server via ssh. I developed a Go api to control our Kubernetes cluster. The Go api serves deployments, pods, namespaces etc.</p><blockquote>With this api, you can restrict the users on the ssh servers and give them permissions only execute specified commands. This makes your servers more secure and you may avoid unexpected situations like deleting a service from kubernetes. This way creates a more secure and controllable interface between users and servers.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ktAi6ahltWRDUCKxM3YtOw.png" /><figcaption>Robo Kube Dashboard</figcaption></figure><p>I created a react app calling Robo Kube that you can manage your Kubernetes cluster like in the screenshot above. By using Robo Kube I’m able to edit the selected pod’s image, restart pod, access to a desired port on a pod with port-forwarding etc. To achieve this, we need to send kubectl commands to our cluster. <br>The given code below gets the deployment information for the given namespace and deployment. With jsonpath option in the command, you can filter the command result and obtain only required fields.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/29a7ad27f5bf89bfb83c50662b4bbe75/href">https://medium.com/media/29a7ad27f5bf89bfb83c50662b4bbe75/href</a></iframe><p>You can see my github <a href="https://github.com/emreekd/gokubernetesapi"><strong><em>repository</em></strong></a>. It is not completed but you can use and change it as you want.</p><p>See you in the next article.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=779a68557826" width="1" height="1" alt=""><hr><p><a href="https://medium.com/hepsiburadatech/control-your-ssh-servers-by-using-a-go-api-779a68557826">Control your Linux servers by using a Go Api</a> was originally published in <a href="https://medium.com/hepsiburadatech">hepsiburadatech</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>