<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Öner Çiller on Medium]]></title>
        <description><![CDATA[Stories by Öner Çiller on Medium]]></description>
        <link>https://medium.com/@onerciller?source=rss-c0bfe3dc155f------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/2*rc8T6dUBuzrsBA2aNZGYrw.jpeg</url>
            <title>Stories by Öner Çiller on Medium</title>
            <link>https://medium.com/@onerciller?source=rss-c0bfe3dc155f------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 06:12:46 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@onerciller/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Re-platforming of Delivery Search: A Scalability Journey of Trendyol Express]]></title>
            <link>https://medium.com/trendyol-tech/re-platforming-of-delivery-search-a-scalability-journey-of-trendyol-express-fba05821a695?source=rss-c0bfe3dc155f------2</link>
            <guid isPermaLink="false">https://medium.com/p/fba05821a695</guid>
            <dc:creator><![CDATA[Öner Çiller]]></dc:creator>
            <pubDate>Fri, 04 Nov 2022 08:04:37 GMT</pubDate>
            <atom:updated>2022-11-04T08:51:52.393Z</atom:updated>
            <content:encoded><![CDATA[<p>Hi everyone,</p><p>In this article, we will talk about the evolution of Trendyol Express delivery search services and the step-by-step transformation from a legacy service to a scalable service.</p><h4><strong>Background</strong></h4><p>As Trendyol grows rapidly, delivery plays a critical role in the company’s growth. Trendyol Express is the logistics company of Trendyol, which grows day-to-day and ships millions of deliveries. It has dozens of management pages and a main Delivery Search Page; please see the following way for more details.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*A9uT2KvfTTvrFarSma0C0g.png" /><figcaption>Delivery Search Page</figcaption></figure><h4><strong>Re-platforming process of the Delivery Search Service</strong></h4><p>There are four main pillars in the process:</p><p>Separation of the delivery search service from the legacy service in the service layer. The following pieces are added:</p><ul><li>Delivery search service</li><li>Replacement of PostgreSQL with Elasticsearch and Couchbase.</li><li>Delivery search BFF (backend for frontend)</li><li>Express delivery query service</li></ul><p><strong>Legacy Service</strong><br>In the early days, the delivery search was handled by the monolithic Hermes API. Since the delivery volume was low, the search mechanism met our needs, and we did not encounter any problems. However, as delivery volume increased rapidly, the response time of the search page increased, and we encountered timeout errors. We needed to scale the system to keep managing it. This is the story of how we decided to separate the delivery service from the legacy service and create a dedicated service.</p><p>The following diagram shows the new structure of the search service: We separated the <strong>delivery search service</strong> from the legacy service, we changed from PostgreSQL to Elasticsearch and Couchbase as database stacks. As a service, the delivery search structure consists of <strong>delivery search service</strong>, <strong>delivery search BFF </strong>(backend for frontend) and <strong>express delivery query service. <br>The express delivery query service</strong> is not a new service, it is another monolithic structure. We use it to retrieve all delivery data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ubb5Bge4qxCUlQ1445XLcw.png" /><figcaption>Delivery Search Structure</figcaption></figure><p><strong>Why do we use both Couchbase and Elasticsearch for searching? <br></strong>Since Elasticsearch is a search engine, not the Database; it shouldn’t be used primarily for data storage. Combination of Elasticsearch and Couchbase provided an efficient search mechanism.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QDBQa-3LgiiMYveZqrgPjg.png" /><figcaption>Elasticsearch and Couchbase mapping examples for delivery search.</figcaption></figure><p>As you can see above, potential search fields are mapped with a unique id<strong> </strong>(deliveryNo) of the ElasticSearch index. When we want to perform a search by mapped fields (e.g. deliveryState), we can retrieve a list of deliveryNos efficiently thanks to the index of Elasticsearch. After having the list of deliveryNos, the<strong> delivery query service </strong>(data source) is called to complete retrieve delivery data.</p><p>Since Couchbase stores data as key-value pairs and can efficiently retrieve values ​​for known keys, we used deliveryNo as the key and used its efficiency.</p><p>Before re-platforming deliver search, the response time was in the range of 2–3sn after that we’ve managed to reduce it to the range of 30–70ms.</p><h4><strong>Re-platforming process of the delivery data source</strong></h4><p>The delivery data source was another bottleneck in our system. We were using the <strong>express delivery query service</strong> to retrieve delivery data (you can see the above delivery search structure diagram). However, it was a monolithic service and connected to a single Kafka topic to handle delivery events.</p><p><strong>A growing team<br></strong>We didn’t have any issues with this service initially as we were a small team and few services were manageable. However, our team grew rapidly, and the team split into smaller teams. Each team had their own microservices under their responsibility, but since the<strong> express delivery query service</strong> is a monolithic structure, each team had to modify the same codebase. Therefore we decided to split the <strong>delivery query service</strong> into multiple services called translator services. Please see the following diagram for the new data source infrastructure.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*083_qXuLQ03euiLUmyj_Hg.png" /></figure><p>As you can see above, each team has a translator service and topics under its own responsibility, and a translator service consists of 2 layers: d<strong>ata-validator </strong>and <strong>delivery contracts.</strong></p><p><strong>Data Validator<br></strong>Each team should be able to interact with the data validator service with their own translator service (i.e., the data validator service needed to be language agnostic). The sidecar application pattern was an out-of-box natural solution. Data validator was written as a sidecar application that runs with the translator service in the same Kubernetes pod. It has a ruleset stored in the Kubernetes config map to decide validation rules and validates data from Kafka topics.</p><p>Please see the following diagram for the data validator application infrastructure.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6REu-CgKlHoMmOrwe2slbQ.png" /></figure><p><strong>Delivery Contracts<br></strong>After validation, we should map the data that come from the topic to <strong>the Delivery Contracts. </strong>Every translator service uses <strong>the Delivery Contracts model. </strong>Therefore, we created a library for it so that able to use shared.<br>After <strong>the Delivery Contracts </strong>mapping, we publish data to <em>earth.delivery.cx.delivery-status-updated.0 </em>topic<em>.</em> Every translator service publishes the same topic.</p><p>Please see the example of the <strong>delivery contract </strong>mapping below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7xLg7vuwze0Xzk4BiXPT6Q.png" /></figure><p><strong>Delivery Detail Consumer<br></strong>the Delivery Detail Consumer is a service that consumes the <em>earth.delivery.cx.delivery-status-updated.0 topic. </em>As I said above, every translator service publishes data on the same topic, and the Delivery Detail Consumer service store the data on the Couchbase.</p><h4><strong>Conclusion</strong></h4><p>As a result, in this significant re-platforming journey, we have come a long way in both the delivery search side and the creation of the delivery data source. We have established a sustainable, scalable and strong structure for The Delivery Search service. We have also built a flexible structure where different teams can manage their own topic events and services with the new delivery data source structure.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fba05821a695" width="1" height="1" alt=""><hr><p><a href="https://medium.com/trendyol-tech/re-platforming-of-delivery-search-a-scalability-journey-of-trendyol-express-fba05821a695">Re-platforming of Delivery Search: A Scalability Journey of Trendyol Express</a> was originally published in <a href="https://medium.com/trendyol-tech">Trendyol Tech</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Configuration Management with Consul and WebSocket]]></title>
            <link>https://medium.com/trendyol-tech/configuration-management-with-consul-and-websocket-3d0f1348f158?source=rss-c0bfe3dc155f------2</link>
            <guid isPermaLink="false">https://medium.com/p/3d0f1348f158</guid>
            <dc:creator><![CDATA[Öner Çiller]]></dc:creator>
            <pubDate>Wed, 19 Feb 2020 06:05:48 GMT</pubDate>
            <atom:updated>2020-02-19T15:30:21.620Z</atom:updated>
            <content:encoded><![CDATA[<p>As Hermes team, we develop logistics and delivery management system for Trendyol. Dashboard, a web application we developed for logistics management, did not have a configuration management on the production environment.</p><p>We could have managed simply configs on ConfigMaps since we have been using Kubernetes for container management. However, we didn’t use ConfigMaps because it also brings a variety of disadvantages. For example, when you set any environment variable to ConfigMaps, you have to restart pods to be able to the activate environment variable. (Unless your application handles config changes like capturing events from k8s in application, or pulling ConfigMaps in a schedule)</p><p>Therefore, as technology we decided to use Consul and WebSocket, because we wanted to dynamically reflect configurations to the Dashboard.</p><h4>Configuration Types</h4><p>We manage two types of configurations:Config and Feature Toggle for the Dashboard.</p><p><strong><em>Config<br></em></strong> Config uses for dynamic configurations such as refresh intervals, page counts and API informations.</p><p><strong><em>Feature Toggle</em><br></strong>One of the safest ways to deploy new releases on a Trunk Based Development (TBD) is to use Feature Toggle. Therefore we deploy every feature with Feature Toggle, eventually, we can manage it on Consul.</p><h4>Overview of Consul and architecture</h4><p>In short, <a href="https://www.consul.io/">consul</a> is a distributed key-value store. Although consul has a lot of features such as service discovery and service mesh, we only use it for config management. Besides, since consul provides web UI, it makes config management easier when a new environment set or update.</p><p>We can ensure that the required services retrieve their configurations from the Consul’s key/value store. The easiest way to do this is to manually add the configuration through Consul’s web UI. Simply by adding a new entry with a key matching the pattern: config/&lt;application_name&gt;/&lt;config_name&gt; you can define a config name with that particular name, for that particular service.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uq1MvC6ZFwh8stSiuRRoyA.png" /><figcaption>Consul UI</figcaption></figure><p>As for the architecture that we built, it is as follows. WebSocket server plays a bridge role in here.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/734/1*s_nQfFzXJ6vLL0U2yQVHXQ.jpeg" /></figure><p>When the consul is updated, the WebSocket server listens for the consul and shares the configs to clients that connected via WebSocket.</p><p>WebSocket Server listens for the changes in consul can look like this:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/fb9efba523b13bcb24042b8747a8ef80/href">https://medium.com/media/fb9efba523b13bcb24042b8747a8ef80/href</a></iframe><p>c.client.KV().List() method that is Golang consul library of the Hashicorp that the request will be blocked until the change occurs in other words, long polling will be performed.</p><h4>Publishing configurations from Consul to Client</h4><p>In Client side, WebSocket Service which is created to connect to WebSocket Server can look like this:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/355634dab9fa5745f754fe2688055e30/href">https://medium.com/media/355634dab9fa5745f754fe2688055e30/href</a></iframe><p>As you see in code, the client connects to the WebSocket server through the open method and then by using the subscribe method that listens for specific events that come from the server.</p><p>For example, in below the client listens for configuration and toggle events using WebSocket Service on the client-side. When the WebSocket server publishes the configurations and toggles, Dashboard will connect through WebSocket Service and it will get all configurations the WebSocket server sent.</p><p>When the client retrieve configurations from the server, it will write all configurations to Local Storage through the updateConfig method.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e1696f2a155a53a4df1fc4df05c61084/href">https://medium.com/media/e1696f2a155a53a4df1fc4df05c61084/href</a></iframe><p>Configurations stored in the browser’s local storage can look like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IxFtpPrH6DATh6tr9nPGOA.png" /><figcaption>Local Storage</figcaption></figure><p>As you see in the following video, we added Feature Toggle v1 and Feature Toggle v2 button to Dashboard. When feature toggle configuration key, FEATURE_TOGGLE_V2_ENABLED, is updated from the consul changes are reflected as realtime.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*f_8WcGhy_prKRq8f72bgUA.gif" /></figure><p>We use React as the main front-end technology and Redux for state management. Therefore, WebSocket messages are dispatched to redux’s store thus configurations are able to be managed easily like the following code.</p><pre>renderFeatureToggle = () =&gt; {</pre><pre>  <strong>if </strong>(<strong>this</strong>.props.toggle.isEnabled(&#39;FEATURE_TOGGLE_V2_ENABLED&#39;)) {<br>    <strong>return </strong>&lt;Button type={&#39;primary&#39;}&gt;FEATURE TOGGLE V2&lt;/Button&gt;;<br>  }<br><br>  <strong>return </strong>&lt;Button type={&#39;danger&#39;}&gt;FEATURE TOGGLE V1&lt;/Button&gt;;<br>};</pre><h4>Conclusion</h4><p>Using Consul and WebSocket, we can manage configurations, by subscribing to changes in that configuration, we are able to change the Configurations and Feature Toggles dynamically in the Dashboard.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3d0f1348f158" width="1" height="1" alt=""><hr><p><a href="https://medium.com/trendyol-tech/configuration-management-with-consul-and-websocket-3d0f1348f158">Configuration Management with Consul and WebSocket</a> was originally published in <a href="https://medium.com/trendyol-tech">Trendyol Tech</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>