<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Mert Şen on Medium]]></title>
        <description><![CDATA[Stories by Mert Şen on Medium]]></description>
        <link>https://medium.com/@mertsenn?source=rss-3ae1a17b85ba------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*LuILbzp7K727C8mtGYy1aQ.jpeg</url>
            <title>Stories by Mert Şen on Medium</title>
            <link>https://medium.com/@mertsenn?source=rss-3ae1a17b85ba------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 00:24:34 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@mertsenn/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Shipping Search Ranking Models Faster: A Config-Driven Approach]]></title>
            <link>https://medium.com/trendyol-tech/shipping-search-ranking-models-faster-a-config-driven-approach-4e55f7e5706b?source=rss-3ae1a17b85ba------2</link>
            <guid isPermaLink="false">https://medium.com/p/4e55f7e5706b</guid>
            <category><![CDATA[search-ranking]]></category>
            <category><![CDATA[search]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Mert Şen]]></dc:creator>
            <pubDate>Thu, 09 Jul 2026 13:50:10 GMT</pubDate>
            <atom:updated>2026-07-09T13:53:22.031Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>Authors:</strong> <a href="https://medium.com/u/3ae1a17b85ba">Mert Şen</a>, <a href="https://medium.com/u/a1f87c221631">Melih Sivri</a>, <a href="https://medium.com/u/830254ab1696">Mert Pınarbaşı</a>, <a href="https://medium.com/u/aee7b85ea5cf">Mustafa Gunaydin</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/0*RXl_qZ4uOvWdIPlL.jpg" /><figcaption>The Triumph of Rapid Iteration: Falcon Heavy’s Dual Landing</figcaption></figure><h3>Introduction</h3><p>In e-commerce search, returning results is only half the job. The other half is returning them <em>in the right order</em>. That is where ranking comes in. After the initial retrieval, we fetch hundreds of features for each product (user behavior, item popularity, price signals, category trends, and more), feed them into an (or multiple) ML model that scores every product, and reorder the results by those scores. What the user sees is a search page ranked by relevance. A better ranking model means a better search experience, which means more conversions.</p><p>We are the Search Relevance team, a mix of developers and data scientists. Our data scientists train the ranking models and decide which features those models need. The developers and ML engineers build and run the serving infrastructure that puts those models into production.</p><p>One thing to make clear up front: none of this is about training models or improving their accuracy. That is the data scientists’ work. Ours is everything that happens <em>around</em> a request to a model, before and after it runs: fetching the right features, assembling them into the shape the model expects, serving them, and getting the model live in the first place.</p><p>And that last part is the bottleneck we focus on. Once a model was trained, getting it into production was a project in itself: configure which features to fetch, describe how they get fed into the model, deploy it. That serving-and-deployment step was a multi-step handoff from a spreadsheet to Go code to a deployment, and every step added friction and time.</p><p>We call the stretch between a fully trained model and that model getting live production traffic its <strong>model lead time</strong>, and shortening it is the whole point of this work. A shorter lead time means more A/B tests in the same window, and more experiments mean faster iterations: we get to see how a ranking change actually performs, learn from it, and try the next idea sooner.</p><p>This is the story of how we rebuilt that step in Trendyol. We went from a world where every new model iteration took days of manual engineering on the serving side, to one where a single YAML file is enough to go live. We call the result our <strong>config-driven ranking platform</strong>.</p><h3>The old world vs. the new world</h3><p>Let’s cut to the chase. Here is what shipping a new ranking model to production used to look like, next to what it looks like now.</p><p><strong>Before: static and code-driven.</strong> The ranking service was a Go application where every model’s configuration (which features to fetch, how to build the inference input, where to send it) was hand-written in a dedicated Go file. When a new model was ready, the data scientists shared a Google Sheet. That sheet was the contract between us: it listed the features the model needed and the exact order they had to be in. We implemented it in Go, wrote tests, registered the model, recompiled, and deployed. A typical model file was over a thousand lines of Go, and the whole path from a trained model to production took two to three days.</p><p><strong>After: config-driven.</strong> Now our data scientists write (or generate) a model signature: a single YAML file with a clear contract that fully describes the model. They push it to the model repository. A pipeline validates the file, opens a merge request in our new ranking service, and we review, merge, and deploy. The same model that used to be a thousand lines of Go is now a 150 to 200 line signature. No manual index mapping. No recompilation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hEYWDcPL3nFB4BPyUVW_9w.png" /></figure><p>The rest of this article is about <em>how</em>: why the old serving loop was so slow, what a ranking request actually does, and the two things we built to replace the old approach. A runtime that reads a model from config, and a delivery pipeline that gets that config into production.</p><h3>The old way, and why it hurt</h3><p>Every iteration followed the same manual loop. We translated the spreadsheet into a dedicated Go package per model version: fetch wiring, dependency chains, and a hand-built inference array where index <em>i</em> had to line up with column <em>i</em> in the training data. Then came unit tests, registration, QA in stage and preprod, cluster wiring, and joint sign-off with the data science team. Any gap (a missing feature group, an empty field, a wrong key) meant going back and waiting until the upstream side was fixed.</p><p>The happy path was <strong>two to three working days</strong> from a trained model to live traffic. That was the model lead time for a single model, and it scaled with complexity. An A/B round is never one model: data science usually wanted two or three challengers at once, each going through the same loop, so a single experiment could eat most of a sprint. Every day spent here was a day we were not learning from the experiment.</p><p>Every step was human-driven. Feature order was typed in by hand, so an off-by-one index or a wrong default could slip past code review. That gap is what pushed us toward a different design.</p><h3>What we wanted</h3><p>After living with that loop, we had a clear picture of what to build, and it was not only about speed. We wanted to change who does what. Data scientists already know which features a model needs and in what order. That knowledge should not have to pass through a manual translation step in Go.</p><p>So the goal became simple: <strong>data scientists should define and ship their own models without writing a line of Go.</strong> Two things had to exist for that to work:</p><ol><li>A <strong>config-driven ranking service</strong> that reads a model’s definition from a YAML file and handles the rest on its own: feature fetching, dependency resolution, inference input construction, and scoring. No hardcoded model logic, no switch statements, no recompilation.</li><li>A <strong>model repository</strong> that data scientists own. They push a YAML, the pipeline validates it, and it flows into the ranking service. We only step in to review and merge which is a very little effort.</li></ol><p>The insight was that most models, however complex, follows the same pattern: fetch features from various sources, assemble them into the right input tensors, send them to the inference server, get scores back. The only things that change between models are <em>which</em> features, <em>from where</em>, and <em>in what order</em>. All of that can be written down declaratively.</p><h3>How a ranking request works</h3><p>Before we get into the config, it helps to see what a request actually does, because that sequence stayed the same the whole time. Only the machinery underneath changed. It starts with the <strong>request</strong>. Search sends a batch of candidate items plus context: who is browsing, what they typed, which region they are in. Nothing is ranked yet, it is just a shortlist worth a second look.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4Chi6IVIkURdH-r_RZ5cIQ.png" /></figure><p><strong>Features</strong> are the signals the model was trained on:</p><ul><li><strong>Product signals</strong>: price band, category fit, recent engagement</li><li><strong>User signals</strong>: size preference, purchase history</li><li><strong>Moment signals</strong>: query intent, device, time of day</li></ul><p>The model does not invent these values. It expects them in a fixed shape, every time.</p><p>Most of that data lives in the <strong>Feature Store</strong> between training and serving: features are computed offline or near-line, stored under lookup keys, and fetched when a request arrives. The ranking service pulls those signals, assembles them into one row per candidate, and sends the batch to the <strong>ML platform</strong>, where the model applies its learned weights and returns a score per candidate. Ranking is just sorting by those scores.</p><p>So the shape of a request never really changed. What changed is everything under it: how the pipeline is configured, and how fast a team can point it at a new model. That is the rest of the story.</p><h3>The model signature</h3><p>Let’s look at what a model signature actually is. It’s a YAML file with three sections: <strong>model info</strong>, <strong>features</strong>, and <strong>inference</strong>.</p><p><strong>Model info</strong> is the easy part. Our models are served on Triton Inference Server and we reach them over gRPC, so all we need is the model name, the gRPC endpoint per environment, and the version.</p><pre>model:<br>  name: mlp-relevance-reranking-v99-bls<br>  url:<br>    stage: mlp-rerank-v99-stage-grpc<br>    preprod: mlp-rerank-v99-preprod-grpc<br>    prod: mlp-rerank-v99-prod-grpc<br>  version: &quot;1&quot;</pre><p><strong>Features</strong> tells the service which feature groups this model needs. Each group has an ID that maps to an entry in the feature catalog (more on that next). Under each group you list the features you want. Those groups might come from completely different data sources, but the data scientist does not need to care. They reference the group ID and pick the features. Where and how to fetch them is handled elsewhere.</p><pre>features:<br>  - id: &quot;AbcdlaQF5PkIQ&quot;<br>    feature:<br>      - &quot;brand_search_ctr_1d_3d&quot;<br>      - &quot;brand_search_ctr_3d_14d&quot;<br>  - id: &quot;52gHQuSkES8E&quot;<br>    feature:<br>      - &quot;category_ctr_3d&quot;<br>  - id: &quot;derived_boolean_features&quot;<br>    feature:<br>      - &quot;is_iphone&quot;<br>      - &quot;is_android&quot;<br>...</pre><p><strong>Inference</strong> is the model’s input contract. It’s an ordered list that tells the service exactly how to build the feature vector sent to the model. Each entry says where to get the value, which feature group it belongs to, and optionally a default if the feature is missing.</p><pre>inference:<br>  - source: &quot;lookups&quot;<br>    name: &quot;brand_search_ctr_1d_3d&quot;<br>    id: &quot;AbcdlaQF5PkIQ&quot;<br>    defaultValue: 0.001668<br>  - source: &quot;lookups&quot;<br>    name: &quot;category_ctr_3d&quot;<br>    id: &quot;52gHQuSkES8E&quot;<br>  - source: &quot;lookups&quot;<br>    name: &quot;is_iphone&quot;<br>    id: &quot;derived_boolean_features&quot;<br>...</pre><p>That’s the whole file, and a data scientist can read, write, and reason about it without touching Go or knowing the service internals.</p><p>The format is more flexible than these small examples suggest. On the input side, it handles many kinds of features, from plain scalar lookups to user activity sequences (the recent items a user viewed or bought, fed to the model in order). On the output side, it is not limited to a single flat vector: a model can declare multiple named tensors, and the service builds each one according to the signature. So the same contract covers everything from a simple linear model to a sequence model with several input tensors. And it keeps growing: when a new requirement shows up, we extend the format without breaking existing models.</p><h3>The feature catalog</h3><p>Here’s a question we had to answer early. If the signature says “fetch feature group AbcdlaQF5PkIQ, how does the service know <em>where</em> to fetch it from and <em>how</em> to build the key?</p><p>That’s the feature catalog. It’s a separate YAML file, a registry of every available feature group. For each group it defines the data source and how to construct the lookup key.</p><pre>feature_groups:<br>  - id: &quot;AbcdlaQF5PkIQ&quot;<br>    provider_type: &quot;feature_store&quot;<br>    lookup_keys:<br>      - source: &quot;request&quot;<br>        field: &quot;itemId&quot;<br>        type: &quot;per-item&quot;<br>      - source: &quot;request&quot;<br>        field: &quot;query&quot;<br>        type: &quot;global&quot;<br>    feature_store:<br>      feature_group_id: &quot;AbcdlaQF5PkIQ&quot;<br>...</pre><p>A few things to unpack.</p><p><strong>Provider type</strong> says where the features live. We support several: Feature Store, a local datastore, and others. The catalog also supports derived features, which are not fetched from anywhere. They are computed on the fly from expressions:</p><pre>- id: &quot;derived_boolean_features&quot;<br>    provider_type: &quot;derived&quot;<br>    lookup_keys: []<br>    derived:<br>      rules:<br>        - output_field: &quot;is_iphone&quot;<br>          source_context: &quot;request&quot;<br>          expression: &quot;request.platform == &#39;IOS&#39;&quot;<br>        - output_field: &quot;is_android&quot;<br>          source_context: &quot;request&quot;<br>          expression: &quot;request.platform == &#39;ANDROID&#39;&quot;</pre><p><strong>Lookup keys</strong> define how the key is built. Each key has a source (where its value comes from) and a type (per-item means one lookup per product, global means a single lookup shared across all products in the request).</p><p>The source can be:</p><ul><li>request: the value comes straight from the incoming request (itemId, userId, query, and so on)</li><li>lookup: the value comes from the output of another feature group, which creates a dependency</li><li>static: a fixed constant, like a region code</li></ul><p>That lookup source is the interesting one. Some feature groups need a value from another group to build their key. To fetch category-level trend features, for example, you first need the item&#39;s category, and that may come from a different group:</p><pre>- id: &quot;cd1fyUviURpw&quot;           # category-level engagement features<br>  provider_type: &quot;feature_store&quot;<br>  lookup_keys:<br>    - source: &quot;request&quot;<br>      field: &quot;query&quot;<br>      type: &quot;global&quot;<br>    - source: &quot;lookup&quot;          # depends on another feature group<br>      group_id: &quot;ks4e6weJ7G52&quot;  # the item feature group<br>      feature: &quot;c&quot;              # the category id it produces<br>      type: &quot;per-item&quot;<br>  feature_store:<br>    feature_group_id: &quot;cd1fyUviURpw&quot;<br>...</pre><p>Here the category group depends on the item feature group because it needs the category (c) that group produces. The service works this out on its own: at startup it reads all the dependencies, builds a graph, and fetches each group as soon as the groups it depends on are done. No manual ordering. In the old system we built these chains by hand in Go. Now they fall out of the catalog.</p><p><strong>Why a separate catalog?</strong> We could have inlined all of this into each signature, but then the same group definitions would be copy-pasted across dozens of models. The catalog is a single source of truth, and it hides the data sources behind one interface. A signature just names a group ID. It never needs to know whether that group lives in the Feature Store, the datastore, or a derived expression. If we add a new source or change how a group is fetched, we update the catalog once and every model gets it for free.</p><h3>Inside the ranking service</h3><p>So a signature says which groups a model needs, and the catalog says how each group is fetched. What turns that into a running model? Three mechanisms let one service host many different models, each with its own feature sources and input shape, without any model-specific code: the <strong>model registry</strong>, <strong>feature providers</strong>, and <strong>dependency resolution</strong>.</p><h3>Model registry</h3><p>In the old design, every model shared the same feature service, provider wiring, and inference client. There was no per-model boundary.</p><p>The model registry fixes that: each model is a self-contained instance, registered under its YAML file identifier at startup.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BiuI6tOAi1fjdpkDEmkeRQ.png" /></figure><p>Registering a model is not just a map entry. When a signature loads, the service:</p><ul><li>resolves each feature group against the catalog,</li><li>builds the model’s <strong>dependency plan</strong> (covered below),</li><li>compiles the inference contract into a matrix or multi-tensor plan,</li><li>and wires up the model’s own feature and inference services with only the providers that model needs.</li></ul><p>This all happens once, at startup. On the request path, the registry just hands back a finished plan to execute.</p><h3>Feature providers</h3><p>Feature providers are the adapters that do the actual fetching. At request time each feature group turns into a <strong>lookup operation</strong>: a concrete call to a provider with resolved keys and a list of fields. For a per-item group, one key is resolved per candidate; for a global group, a single key is resolved once and reused. The provider returns documents, and the values are written into a shared result that later groups and the inference step read from.</p><p>Providers sit behind a common interface, so they are pluggable, and a model registers only the providers its groups need. The catalog is shared, but the provider wiring is per model. The service also keeps track, per item, of which key was used for each group, and that is what lets a later group build a key from an earlier group’s output, like combining an item’s id with its fetched category.</p><h3>Dependency resolution</h3><p>A model can reference dozens of feature groups, and not all of them can be fetched at once. Some depend on values produced by others: a user-by-category feature needs the item’s category, a stock feature might need the user’s size, which itself depends on item attributes. Fetching everything one group at a time is correct but slow. Fetching everything at once is fast but wrong. What we want is the middle: run each group the instant its dependencies are ready, and run everything else in parallel.</p><p>So we model the feature groups as a <strong>DAG</strong>. Each group is a node, and an edge means “this group needs a value from that group”, taken straight from the lookup keys in the catalog. When a model loads, the service builds a <strong>dependency plan</strong> from that graph: for each group, the set of parent groups it waits on. It also detects cycles and rejects them right there, so a bad catalog never reaches production. The plan is stored on the model in the registry, so no request ever recomputes it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jy6_ZErRo-QZCRKxGqgyMA.png" /></figure><p>The DAG drops the batches. At request time the service launches <strong>one worker per group, and each worker waits only for its own parents</strong> before fetching. FeatureGroup1 and FeatureGroup2 have no dependencies, so they start immediately. FeatureGroup3 and FeatureGroup4 wait only for FeatureGroup1, and FeatureGroup5 waits only for FeatureGroup3, so a slow FeatureGroup2 blocks nobody. The critical path is now the longest <em>real</em> dependency chain (here FeatureGroup1 → FeatureGroup3 → FeatureGroup5), not the slowest batch.</p><p>Put together, these three swap model-specific code for configuration plus one fixed execution engine. A new model version is just a new registry entry: a signature in the model directory, its groups in the catalog, providers already available. The service binary does not change.</p><h3>The model delivery pipeline</h3><p>The runtime is only half of it. Getting a signature <em>into</em> it, safely, is the other half, and it is where the days-to-hours jump really comes from. The whole thing lives in the <strong>model repository</strong>, deliberately separate from the ranking service: <strong>data scientists own the configuration, the ranking service owns execution.</strong></p><p><strong>The starting point.</strong> The model must already be trained, registered, and deployed on the ML platform, with its features in the online store. From there, one merge request drives everything. A data scientist runs a generator script that turns their feature spreadsheet into a signature (resolving catalog dependencies along the way) and opens the MR. No engineer is in the loop until the very end.</p><p><strong>What the pipeline does</strong>, end to end, in <strong>15 to 20 minutes</strong>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NUsGua_meGCzR7TuQ2ZuYQ.png" /></figure><ol><li><strong>Validate</strong>, and stop on any failure: YAML syntax, duplicate features, every feature checked against the live Feature Store, lookup keys checked against the feature platform, and service URLs that actually resolve.</li><li><strong>Build</strong> a Docker image on a fresh branch of the ranking service repo.</li><li><strong>Feature test</strong>: deploy the candidate configuration and checks feature and model parity. This is the check that used to block QA for days.</li><li><strong>Onebox</strong>: deploy an isolated sandbox instance and post its URL to Slack with a ready-to-run request, so data scientists can fire live traffic at the exact build and sanity-check the ranking before it touches a shared environment.</li><li><strong>Quality checks</strong>: automated ranking-quality checks run against the onebox.</li><li><strong>Open the MR</strong>: tear the onebox down and open the final merge request in the ranking service.</li></ol><p>From there we pick it up: review the diff, check the A/B test mappings, do the model scale if needed, then merge and deploy. Data scientists drive the model all the way to a tested, live-verified onebox on their own. Our part is a quick review and the final rollout. Neither side writes serving code anymore.</p><h3>Wrapping up</h3><p>We set out to shorten <strong>model lead time</strong>, the gap between a trained model and that model serving live traffic. What we built is a config-driven ranking platform in two parts: a runtime that executes any model from a signature and catalog, and a delivery pipeline that validates and ships that config without anyone writing code.</p><p>We have already shipped many ranking iterations through this system, and the difference is real. Model lead time dropped from two to three days to hours. A thousand lines of hand-written Go became a YAML file a data scientist can own and verify on a onebox before it touches a shared environment. Errors that used to slip through code review now tend to show up in validation or on the onebox, not in production.</p><p>It is still evolving. The signature format keeps growing, the catalog keeps absorbing new data sources, and the pipeline keeps picking up checks that used to be manual. But the loop we were trying to unlock is already there: shipping a model is a merge request, not a sprint-long project, so we can run more A/B tests and learn from our ranking models faster.</p><p>The foundation is in place. <strong>Models are configuration, not code.</strong> The ranking service stays the same; what changes is the YAML.</p><p>We barely touched the testing side of the pipeline here: feature tests, the onebox sandbox, and the quality checks that run before anything reaches a shared environment. That deserves its own write-up, and it will be the subject of a follow-up article.</p><h3>About Us</h3><p>We’re building a team of the brightest minds in our industry. Interested in joining us? Visit the pages below to learn more about our open positions.</p><p><a href="https://careers.trendyol.com/">Home - Trendyol Careers</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4e55f7e5706b" width="1" height="1" alt=""><hr><p><a href="https://medium.com/trendyol-tech/shipping-search-ranking-models-faster-a-config-driven-approach-4e55f7e5706b">Shipping Search Ranking Models Faster: A Config-Driven Approach</a> was originally published in <a href="https://medium.com/trendyol-tech">Trendyol Tech</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Unveiling Rerank: Trendyol’s Approach to a Tailored Search Experience]]></title>
            <link>https://medium.com/trendyol-tech/unveiling-rerank-trendyols-approach-to-a-tailored-search-experience-c6f3c1e69b73?source=rss-3ae1a17b85ba------2</link>
            <guid isPermaLink="false">https://medium.com/p/c6f3c1e69b73</guid>
            <category><![CDATA[searchrelevance]]></category>
            <category><![CDATA[search]]></category>
            <category><![CDATA[reranking]]></category>
            <category><![CDATA[trendyol]]></category>
            <category><![CDATA[personalized-search]]></category>
            <dc:creator><![CDATA[Mert Şen]]></dc:creator>
            <pubDate>Tue, 30 Jul 2024 09:34:57 GMT</pubDate>
            <atom:updated>2024-07-30T09:34:57.295Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*e-TTpOlFI9U9drqJUjGcIA.jpeg" /></figure><p>In the realm of e-commerce, delivering personalized and relevant search results is crucial for both user satisfaction and the success of a large-scale company like Trendyol. As the Search Relevance Team, our mission is to enhance the search experience by ensuring users are efficiently presented with the most pertinent and personalized search results. We aim to assist users in quickly finding products that align closely with their interests and requirements.</p><p>Within the Search Relevance Team, developers and data scientists work closely to achieve these objectives.</p><p>This article aims to explore Rerank, a central project within our team, and how it empowers us to deliver personalized and relevant search results to Trendyol users.</p><h3>Search Before Rerank</h3><p>Before delving into Rerank, it’s important to understand how Search at Trendyol performed before the introduction of the Rerank project. When a user initiates a search, the process initially involves Matching, followed by the Relevancy and Smartlisting steps.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/509/0*vgpJseadEaeJ7_KG" /></figure><p><strong>Matching:</strong> The initial phase of our search algorithm involves matching products to user queries, and identifying products based on their relevance to the provided search terms. Currently, we use exact matching between the product’s title, category, brand, and attribute information and query to retrieve products related to a given query from Elasticsearch.</p><p><strong>Relevancy:</strong> Positioned as the second stage, the relevancy algorithm evaluates the alignment between a specific query and each product category. This algorithm leverages click distributions to calculate category relevance scores. For instance, in an “iPhone” search, smartphones are ranked higher due to the application of category boosting with Relevancy. Without category boosting, products like phone cases with higher metrics such as sales and impressions would appear at the top.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1009/1*Qd8rMGV891Xk8WS0Uuz8iQ.png" /><figcaption>Category Boosting in “iPhone” search</figcaption></figure><p>Gender boosting is also applied in the Relevancy phase to refine the sorting process with query-specific insights, thereby minimizing the potential for displaying irrelevant results. For example, in a search for “t-shirts”, gender boosting is applied through Relevancy. As a result, male users are shown men’s t-shirts more prominently, while female users see women’s t-shirts ranked higher. Thus, Products from the matching phase are ranked by relevant category and gender information if applicable.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1010/1*e_xajN8I7Up6nyRFs0Eeow.png" /><figcaption>Gender boosting example for a “t-shirt” search of a male user</figcaption></figure><p><strong>Smartlisting:</strong> As the third step, the same-category products are ordered by smartlisting score. The order/impression prediction of the Smartlisting score is based on a predictive algorithm. Although this method works well for capturing the overall performance of a product, it may not adequately assess how relevant a product is to individual searches and user preferences.</p><p>These steps (Matching, Relevancy, and Smartlisting) remain intact, with Rerank incorporated after them (Matching, Relevancy, and Smartlisting followed by Rerank). While the Matching, Relevancy, and Smartlisting phases occur within the Search API, the Rerank process is handled within a distinct project named the Rerank API.</p><h3>The Need for Rerank</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/0*JA61sQ_Tp4_Ps1so.jpg" /></figure><h4>Need for Personalization</h4><p>No personalization is applied before Rerank. We aim to display search results personalized by incorporating user-related features along with Rerank. Our goal is to prioritize products that align with user preferences based on their historical behavior and demographic information through search results.</p><h4><strong>Overall performance focus</strong></h4><p>While Smartlisting effectively measures the global performance of products, it may fall short of capturing the nuanced relevance of products to specific user queries and users&#39; intent for a particular product. This singular focus can lead to the dominance of high-performing products across various queries, overshadowing potentially more relevant and personalized products.</p><p>The current method of calculating category relevance scores unintentionally creates category bias. This bias arises from the overemphasis on certain categories during searches, resulting in a skewed and incomplete representation of relevant options. This can cause our relevancy algorithm to become fixated on popular categories, neglecting a broader range of potentially relevant items.</p><p>Additionally, the existing relevancy logic sometimes results in only one type of product being highlighted, limiting the effectiveness of category-based relevance. Consequently, for specific queries like “kırmızı elbise” (red dress), category relevance may not accurately reflect user needs. From a user’s perspective, featuring unrelated products prominently can be frustrating and may lead to dissatisfaction, such as showing “socks” at the top of a general “clothes” search.</p><p>Relying solely on Smartlisting scores and the limitations of category-based relevancy can result in irrelevant search results, affecting overall user experience and satisfaction.</p><p>Therefore, we need Rerank for a better experience of personalization and query relevance.</p><h3>How Rerank Works</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/813/0*UhWW1gqQ_PlTBShj" /></figure><p>With the Matching, Relevancy, and Smartlisting steps, products are already ranked. If Rerank is not applied, products will be displayed in the order of this ranking. The top N = 1000 products are called the candidate products for Rerank. Rerank just re-ranks those candidate products and determines the final display order of products.</p><blockquote>Rerank not only helps display personalized results but also enhances the relevance of displayed products to the query by utilizing interaction features between query terms and products.</blockquote><p>Let’s say, with the Matching, Relevancy, and Smartlisting steps, the candidate product ID list was created as follows: 1, 2, 3, 4, 5. In the absence of Rerank, the order that will be shown to the user is <strong>1, 2, 3, 4, 5</strong>. However, with Rerank, this order can change completely and for 5 products, 120 different sortings may exist. Each product is assigned a score by Rerank and subsequently arranged from highest to lowest score. For instance, the reordered list after applying Rerank might be <strong>5, 2, 1, 3, 4</strong>.</p><p>Consequently, the search results are presented in the order determined by Rerank. Therefore, Rerank is a crucial step as it determines the final order of the products that will be displayed to the user.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/587/0*ko5quXtoA8Tsjvnv" /></figure><p>We have described Rerank as a black box. Internally, Rerank employs a Machine Learning model for product ranking. This model can take any 1+ sized sets of &lt;user, query, item/brand/category/other derivatives&gt; features as inputs and generates a prediction score for each item. The prediction scores are then used to establish the final order of products. Rerank can also utilize multiple ML models. An equation integrating prediction scores from various models for the same products can be employed to compute the final scores of the products.</p><p>In the current system, following the Matching, Smartlisting, and Relevancy stages, the top 1000 highest-scored products are forwarded to Rerank from the Search API. Therefore, Rerank processes and reranks a maximum of 1000 products. If a user scrolls through more than 1000 products during a search request, the remaining products are displayed in the order determined before Rerank is applied.</p><p>Although search API gets reranked results for 1000 products, the user will see only a portion of it in the UI at a time. When the user starts scrolling, another search request is made to Search API. Then, does the Search API send a request to Rerank again? No, In Search API, for a user’s search, reranked products are stored in a distributed cache for 10 minutes with a key that can identify a specific search journey of the user, which we call the <strong>Unique Search Journey Key</strong>. Thus, when the user scrolls, Search API uses the ranking in the cache instead of repeatedly requesting the rerank.</p><h4>An Example Flow of a Search Request with Phases</h4><ul><li>An illustrative flow of a search request unfolds as follows: Upon entering the query “kırmızı elbise” (red dress) on the search box, the system initiates retrieval of 35000 products during the matching phase. Products categorized as “Dress” undergo category-specific refinement to enhance result relevance. Subsequently, the reranking phase focuses on the top 1000 products with the highest adjusted ratings.</li><li>During reranking, the system leverages different features to optimize the sortings of displayed products further. This process aims to prioritize highly relevant and personalized products, optimizing user experience by presenting products tailored to the specific search query and individual preferences.</li><li>Finally, products are displayed in the order Rerank’s output determines for the “kırmızı elbise” search.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Q-DrC8xTds61ORuoRUdbgQ.png" /></figure><h4>Separation of Rerank by Services</h4><p>In the search results on Trendyol, certain products are marked with a “sponsored product” badge, such as the third product in the image below. Internally, we refer to these as “ads.” While ads are also products themselves, we distinguish them as ads to denote that they are sponsored products.</p><p>Rerank handles products and ads separately. Specifically, products undergo ranking through a distinct request to a separate endpoint as <strong>/rerank-products</strong>, whereas ads are processed via individual requests to their own <strong>/rerank-ads</strong> endpoint. Thus, for the image below, in Rerank, the 1st, 2nd, and 4th products are ranked against each other and other products, while the 3rd and other ads products are ranked separately among themselves.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/991/0*0rSSjkIvDsmlBnKr" /><figcaption>A search result showing an ad: “sponsored product”</figcaption></figure><p>Ads and Product Reranks are handled through separate requests due to their distinct requirements and objectives. The ML models used for Reranking Ads and Products, along with their feature sets, are tailored differently to meet specific targets. For example, one model may focus on enhancing the Click-Through Rate (CTR), while another may aim to improve the Conversion Rate (CR). Additionally, our services will extend beyond Ads and Products such as to include Reranking on flash-sale pages, which will involve yet another ML model and a separate request process.</p><p>Presently, Rerank is only operational in Turkey. However, we aim to extend its application to other international markets such as GULF, CEE, and more. This expansion will also necessitate the use of different ML models customized for each market’s unique requirements and preferences.</p><blockquote>Every business and market has unique needs and objectives, so they require different machine learning models and targets.</blockquote><h3>High-Level Architecture of Rerank</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/916/0*EEQ2DWHaUxBj9yBe" /></figure><p>Initially, our client search API transmits the product IDs to Rerank within the request body. Subsequently, the Rerank API executes Lookup operations, which involve retrieving necessary features from the Feature Store for each item for the ML model. The Feature Store, maintained on a NoSQL Database, serves as our repository for these features. Features are extracted and prepared to accommodate the model’s requirements for each item. These features are then forwarded to the <a href="https://developer.nvidia.com/triton-inference-server">Nvidia Triton Inference Server</a> for an on-the-fly inference process.</p><blockquote><strong>Inference</strong> is the process of using a trained machine learning model to make predictions or draw conclusions from new data.</blockquote><p>The inference request sent from Rerank API to Triton is structured such that each array corresponds to the features of an individual item as follows:</p><pre>[<br>  [&lt;feature1&gt;, &lt;feature2&gt;, &lt;feature3&gt;, …], <br>  [&lt;feature1&gt;, &lt;feature2&gt;, &lt;feature3&gt;, …], <br>  …<br>]</pre><p>The output Rerank API gets from Triton is roughly as follows, which is the prediction score for each item:</p><pre>{<br>  ...<br>  &quot;data&quot;: [0.5230713242013007404, 0.4662513542218167203, ...]<br>  ...<br>}</pre><p>Following inference, a post-processing step occurs to refine results before they are returned to the Rerank API. Subsequently, the prediction score for each item is sent back to the Rerank API as the output. The Rerank API then arranges products in descending order based on these prediction scores before responding to the Search API. Ultimately, products are displayed in search results according to the sorting conducted by the Rerank process.</p><h4><strong>How Do We Serve ML Models?</strong></h4><p>Our ML models are deployed using Triton. <a href="https://developer.nvidia.com/triton-inference-server">Triton Inference Server</a>, developed by NVIDIA, is an open-source, high-performance software designed for serving machine learning models in production environments.</p><p>We have used LightGBM and XGBoost as our Rerank ML models so far. We use Triton’s Business Logic Scripting (BLS) with the models. <strong>Business Logic Scripting (BLS)</strong> functions as a Python backend that operates immediately prior to and following the inference of the ML model(s). BLS preprocesses features before forwarding them to the ML model for inference. It also facilitates post-processing of the model’s inference results, enhancing our ability to refine outputs effectively. Pre-process and post-process logics also enable us to use and orchestrate ensembled models together.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/552/0*KAO_6F1uExPD43M4" /></figure><h4><strong>What Features do ML models Use When Determining Prediction Scores?</strong></h4><p>Models utilize a variety of features including:</p><ul><li><strong>Item Characteristics</strong>: Gender, price, review, item order gender ratio, content price group information, and more.</li><li><strong>User History</strong>: Brand and category propensities, demographics, etc.</li><li><strong>Search Query Data</strong>: Aggregated information from search queries.</li><li><strong>Query Interactions</strong>: Features such as Query x Item, Query x Category, Query x Brand, and Query x Category x Brand in different timeframes like Last 1 Hour, Last 1 Day, etc. including metrics like clicks, CTR, and impressions.</li></ul><p>Features are updated in scheduled batches like every 10 minutes, 1 hour, 24 hours, etc., and obtained mostly via clickstream data. User and query-related features sent to the inference server are common for each item in an individual request. To enhance the effectiveness of Rerank, our Data Science team continuously iterates ML models, and we aim to increase the feature diversity and efficiency.</p><h3>How Do We Ensure Rerank’s Success?</h3><blockquote>The fundamental success criterion for Rerank is an improvement in user actions, such as conversion rates (CR), click-through rates (CTR), and attributed actions.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/480/0*GVgw6FEdm6-ZS84V.gif" /></figure><p>At Trendyol, A/B testing is extensively employed to compare two versions of a webpage or app to determine which one performs better concerning a specific metric, such as user engagement or conversion rates. By assigning users to either version A or version B, we can measure the impact of changes and make data-driven decisions.</p><p>To initially measure the success of Rerank, we conducted an A/B test where group A did not have Rerank applied, while group B did. Observing a significant improvement in business metrics for group B, we enabled Rerank for all users.</p><p>Our Data Science team continuously iterates on the machine learning models. When introducing a new ML model, we conduct further A/B testing to evaluate its performance against the current model. If the new model demonstrates significant improvement based on A/B test results, we replace the existing model to serve the Rerank with the new model for all users. Occasionally, we test multiple new models simultaneously, employing a sort of A/B/C/D testing.</p><p>Some metric results from the testing of the recent Rerank Ads model iteration:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/271/0*02WDjpZl9QR1kCD_" /></figure><p>Below you can see a few examples of how we can show more relevant &amp; personalized results with Rerank:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*cCRPJ3eTPazluaWS" /><figcaption>An example of displaying more relevant results thanks to query-item relevance</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*60Blpdk00Q4iruG1nBKrtw.png" /><figcaption>An example of displaying personalized results with Rerank-Ads</figcaption></figure><h3>Some Challenges of Rerank</h3><ul><li><strong>Maintaining Query Relevance in Personalization:</strong> While enhancing product results with personalization, it is essential to retain their relevance to the search term, necessitating the selection of appropriate targets and features.</li><li><strong>Predicting Performance for Low-Impression Products:</strong> Accurately estimating the performance of products with low impressions is crucial due to the lack of user interaction data. Correct predictions in model simulations are therefore imperative.</li><li><strong>Improving Candidate Selection:</strong> The selection of candidate products for Rerank can be optimized. The more effectively candidates are chosen, the greater the potential for Rerank to succeed. For instance, while Rerank can elevate the position of the 1000th product to the top, it cannot do the same for the 1001st product, even if it would yield a more personalized and relevant result for the user’s search, simply because it was not included in the candidate products list.</li><li><strong>Position Bias: </strong>Products displayed on top of the search are prone to have high metrics such as impressions and clicks. That is why, the model may experience position bias. DS is working to address and mitigate this issue within the Rerank model.</li><li><strong>Using one ML Model for All Categories: </strong>We use a single Rerank model for all categories, which requires comprehensive feature engineering to address the diverse needs of various categories. Handling this broad range of categories effectively is a complex task.</li></ul><h3>Future Plans</h3><h4><strong>Utilizing Vectors in Search Relevance: Moonshot and Prerank</strong></h4><p>We aim to implement a two-tower architecture through the Moonshot and Prerank projects, leveraging user and item vector embedding similarities to enhance search results presentation.</p><blockquote>The two-tower architecture is composed of two distinct towers: one dedicated to the user and the other to the item. Utilizing deep neural networks, it learns high-level abstract representations for both users and items based on their past interactions. The output measures the similarity between the user embedding and the item embedding, indicating the user’s interest in the given item.</blockquote><p><strong>Moonshot</strong>: Integrating vector embedding similarity as a feature in the Rerank process.</p><p><strong>Prerank</strong>: Improving the candidate pool selection of products before Rerank.</p><p>We have started testing these projects and are refining them through iterative improvements. The iterations are divided into phases, and upon completion, we plan to phase out the Smartlisting and Relevancy algorithms. The ultimate search process is expected to follow the steps of <strong>Matching -&gt; Prerank -&gt; Rank -&gt; Rerank</strong> (with a smaller candidate size), utilizing user-query and item vector embeddings at each stage.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/0*Ah6u3oJ-gjgffHap.jpg" /></figure><h4><strong>Using Real-time Features in Rerank</strong></h4><p>The features currently stored in our Feature Store are updated in scheduled batches. However, for more accurate prediction scores in Rerank, we aim to incorporate real-time user features such as visits, favorites, baskets, and orders. This approach will enable us to provide more real-time predictions.</p><p>The ultimate point we want to reach is to achieve event-based and real-time request predictions. Through these iterations, we aim to establish a high-throughput, low-latency infrastructure for real-time personalization.</p><h4><strong>Expanding Rerank’s Sphere of Influence</strong></h4><p><strong>Include Anonymous Users: </strong>Currently, only logged-in users can see Reranked results. However, we aim to extend this capability to anonymous (logged-out — userID is unknown) users as well for better relevancy and personalization. Even if the user is anonymous, we aim to apply personalization to some extent still, using their in-session user activity features.</p><p><strong>Rerank in Different International Markets: </strong>Presently, Rerank is active only in Turkey. However, given Trendyol’s significant growth in the international market, we aim to enable Rerank in regions such as the GULF, CEE, AZ, and DACH markets. This expansion will allow us to deliver more personalized and relevant search results, thereby enhancing our business metrics in these areas.</p><p><strong>Rerank in More Context: </strong>Currently, Rerank is enabled only on the main search page. However, we plan to extend Rerank to additional pages, such as Flash Sales and Homepage Widgets that are empowered by search behind the scenes. By doing so, we can apply Rerank to a wider range of search results, providing our users with a more relevant and personalized search experience.</p><h4>Unified Platforms for Feature and ML Model Serving</h4><p>We have developed and managed our Feature Store and handle the serving and maintenance of ML models for inference as stated. Currently, we are collaborating with ML Platform and Feature Store teams to develop unified platforms for Feature and ML Model serving that can be utilized across Trendyol. Our goal is to transition these platforms to the relevant teams, thereby enhancing the productivity of both Data Scientists and Developers and contributing to the internal ecosystem.</p><h3>Conclusion</h3><p>In this article, I have tried to cover Rerank, a project developed by the Search Relevance team to deliver more personalized and relevant results to our users. Additionally, I have outlined some of the future work we plan to undertake in the next iterations. If you are interested, you can watch the <a href="https://www.youtube.com/watch?v=xfB12mh2vi4&amp;ab_channel=TrendyolTech">meetup</a> where our Data Science team discusses Rerank and other projects we are working on. Stay tuned for upcoming articles on interesting topics from the Search Relevance team!</p><p>Hope you enjoyed reading this article. Thank you for your time. Please do not hesitate to <a href="https://www.linkedin.com/in/mert-%C5%9Fen-a507181b3/">contact me</a> with any questions or feedback.</p><h3>Join us</h3><p>We’re building a team of the brightest minds in our industry. Interested in joining us? Visit the pages below to learn more about our open positions.</p><p><a href="https://careers.trendyol.com/">Home - Trendyol Careers</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c6f3c1e69b73" width="1" height="1" alt=""><hr><p><a href="https://medium.com/trendyol-tech/unveiling-rerank-trendyols-approach-to-a-tailored-search-experience-c6f3c1e69b73">Unveiling Rerank: Trendyol’s Approach to a Tailored Search Experience</a> was originally published in <a href="https://medium.com/trendyol-tech">Trendyol Tech</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>