<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[syte_engineering - Medium]]></title>
        <description><![CDATA[Syte is the world’s first Product Discovery Platform for retail. We use visual AI to create intuitive search and discovery experiences for all types of shoppers. On this blog, our R&amp;D Team shares the behind-the-scenes of building Syte’s unique technology. - Medium]]></description>
        <link>https://medium.com/syte-engineering?source=rss----2fbf22bb84ad---4</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>syte_engineering - Medium</title>
            <link>https://medium.com/syte-engineering?source=rss----2fbf22bb84ad---4</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 10:03:09 GMT</lastBuildDate>
        <atom:link href="https://medium.com/feed/syte-engineering" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Optimizing ETL Processes: Exploring the Shift From VMs to Kubernetes at Syte]]></title>
            <link>https://medium.com/syte-engineering/optimizing-etl-processes-exploring-the-shift-from-vms-to-kubernetes-at-syte-1bf7c9cf9214?source=rss----2fbf22bb84ad---4</link>
            <guid isPermaLink="false">https://medium.com/p/1bf7c9cf9214</guid>
            <dc:creator><![CDATA[Ilya P]]></dc:creator>
            <pubDate>Mon, 24 Jul 2023 13:29:37 GMT</pubDate>
            <atom:updated>2023-07-24T13:29:37.712Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sATo9EeHa195akoEXSWosA.jpeg" /></figure><p>Authors: Sapir Bitton and Ilya Pocherk</p><p>Data is the lifeblood of modern eCommerce companies. Without it, businesses lose insights into customer behavior, market trends, and product performance. But processing and understanding this vast ocean of data is no small feat. At Syte, we are experts in tackling this challenge, particularly for eCommerce companies without dedicated engineering teams.</p><p>One significant part of our solution is data ingestion, which takes our clients’ raw data, transforms it into a useful format, enriches it with further information, and loads it into our database. This process is often called ETL — Extract, Transform, and Load. Previously, our data ingestion process ran on a fleet of Amazon EC2 (Elastic Compute Cloud) instances. But now, we’re looking for a different approach and found it in the Kubernetes-native solution, Argo Workflows.</p><h3>Challenges of Running on Virtual Machines</h3><p>Our previous ingestion process had a specific launch template, and we maintained a fixed number of workers for each step. This architecture was our standard for a while, but we noticed it presented several disadvantages over time:</p><ul><li><strong>Variable computational requirements</strong>: The jobs’ computational requirements varied significantly from step to step. Because of this, there were numerous specifications for each stage, complicating the overall process and making it more challenging to manage.</li><li><strong>High resource usage</strong>: The system was continuously running, preventing us from optimizing processing times and using computational resources efficiently. This led to a high level of resource waste between pipelines.</li><li><strong>Internal development demands</strong>: The visibility of the process depended entirely on an in-house UI that required constant development and maintenance. This reliance created additional work for our development team and posed challenges in achieving optimal troubleshooting and user experience.</li><li><strong>Inability to quickly adjust to needs</strong>: Dependency on Ansible playbook resulted in slower-than-ideal startup times at application launch. This slowdown hampered our agility and our ability to meet rapidly changing needs.</li><li><strong>Lack of interchangeability</strong>: Using the “pets, not cattle” application pattern exacerbated troubleshooting difficulties.</li></ul><p>We considered using EC2 Spot fleets to address some of these issues. However, the possibility of spot interruptions potentially freezing the pipeline deterred us from this path. In short, our old system, while functional, had numerous inefficiencies.</p><h3>The Power of Argo Workflows and Kubernetes</h3><p>Recognizing these challenges, we began exploring more efficient, scalable, and reliable solutions for our ETL process. We found the answer in Argo Workflows, an open-source, container-native workflow engine for orchestrating parallel jobs on Kubernetes.</p><p>Argo Workflows allows us to define workflows where each step is a container. It provides the flexibility to model multi-step processes as a sequence of tasks or as a Directed Acyclic Graph (DAG). This functionality is particularly valuable for complex processes like ours, which can involve up to twelve steps depending on client specifications and needs.</p><p>In addition to providing a robust framework for complex workflows, Argo Workflows enables us to run compute-intensive jobs for machine learning or data processing more efficiently. This is achieved by orchestrating parallel jobs on Kubernetes, which substantially reduces the time needed for these operations compared to our previous VM-based system.</p><h3>Constructing the New ETL Workflow</h3><p>As we adopted Argo Workflows, we created a base workflow containing several template specifications for different step types, such as debugging, EFS (Elastic File System) preparation, and task execution. Each of these step types has its own specifications, such as CPU and RAM requirements, retry strategies, and Docker image definitions. Using these template specifications, we can generate specific workflows according to each client’s configuration, creating a bespoke processing pipeline for every customer.</p><p>Our new system operates within a Kubernetes environment, with computing power provisioned by Karpenter. This architecture allows us to use spot capacity as much as possible, taking full advantage of lower-cost compute resources. And in times of high demand, we can quickly increase capacity, ensuring our processes remain uninterrupted.</p><p>The significant shift from our old architecture is the move away from a static number of workers. Now, workers are dynamically added or removed based on demand and processes complete faster, with reduced wait times and congestion for clients.</p><h3>Benefits of the New Implementation</h3><p>This updated architecture has numerous advantages — from cost savings to improved processing times. Let’s explore some of these advantages:</p><ul><li><strong>Cost optimization: </strong>By shifting to Argo Workflows, we managed to cut our costs by 30% compared to the previous infrastructure set-up. We achieved this by using spot instances more effectively and reducing idle computing capacity. To enhance this approach, we are relying on AWS cost allocation and Kubecost for better budget planning and optimization.</li><li><strong>Enhanced customer experience: </strong>Our new ETL process offers better timing for each customer, which leads to faster results. It also improves security and provides an isolated environment during processing. Being a heterogeneous solution based on Kubernetes, it can also run in a hybrid cloud environment, offering flexibility and scalability.</li><li><strong>Improved development and operations: </strong>Argo Workflows provides our development and operations teams with a much-improved user experience. With a more intuitive and detailed UI, we can better monitor and manage our workflows. The rich feature set of the Kubernetes ecosystem, including tools like Prometheus, Grafana, logs, and debug, further enhances our development and management experience.</li></ul><p>Because we were impressed with the Argo Workflows ecosystem, we decided to move our Kubernetes-based cron jobs to AWS Cron Workflows and are pleased to have all the functionality, visibility, and transparency of Argo Workflows UI. Now, we can easily check job statuses, schedules, and logs, and a simple UI click helps us to duplicate any job. Our new ETL process also enables a better and faster CI/CD process, and by using ArgoCD instead of Ansible, we’ve dramatically reduced application deployment times.</p><h3>Techniques Integrated by Syte</h3><p>During the transition to this new system, wee also implemented several specific techniques to maximize efficiency and robustness:</p><ul><li><strong>Debug pod:</strong> We run a debug pod to provide the best troubleshooting experience when needed. This feature is based on the Argo workflow “on exit handler,“ which is triggered when a pipeline fails.</li><li><strong>EFS sharing:</strong> EFS enables data sharing between pipeline steps. We’ve prepared multiple EFS disks and related Kubernetes storage classes. By randomly selecting one during workflow creation, we circumvent known EFS access point limitations.</li><li><strong>Custom metrics and labels:</strong> We use custom metrics and Kubernetes labels to provide more vital information in Prometheus monitoring.</li><li><strong>Semaphore usage:</strong> We use the “semaphore” feature to have fine synchronization control when we need to limit the parallel execution of specific steps.</li></ul><h3>In Summary</h3><p>Transitioning our ETL process from a VM-based system to Argo Workflows running on Kubernetes has improved our operations at Syte. We’ve overcome the challenges of our old system, significantly cut costs, and improved our processing speeds — all while offering a better experience for both our clients and our internal teams.</p><p>By embracing the power of modern container-native workflow engines in managing complex ETL processes, we’ve solved our immediate challenges and laid a strong foundation for future growth and innovation.</p><p>We plan to continue iterating on our ETL infrastructure to optimize our processes further. As we learn more about Argo Workflows, Kubernetes, and other technologies, we’ll continue refining our strategies to provide efficient and profitable services to our customers.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1bf7c9cf9214" width="1" height="1" alt=""><hr><p><a href="https://medium.com/syte-engineering/optimizing-etl-processes-exploring-the-shift-from-vms-to-kubernetes-at-syte-1bf7c9cf9214">Optimizing ETL Processes: Exploring the Shift From VMs to Kubernetes at Syte</a> was originally published in <a href="https://medium.com/syte-engineering">syte_engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Python Code Quality Checks]]></title>
            <link>https://medium.com/syte-engineering/python-code-quality-checks-7348b1bb0289?source=rss----2fbf22bb84ad---4</link>
            <guid isPermaLink="false">https://medium.com/p/7348b1bb0289</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[code]]></category>
            <dc:creator><![CDATA[Maor Nissan]]></dc:creator>
            <pubDate>Tue, 03 Jan 2023 10:10:23 GMT</pubDate>
            <atom:updated>2023-02-27T07:52:38.439Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dvOquG2PJ7prpFrzuisG9Q.jpeg" /><figcaption>Photo by Maor Nissan</figcaption></figure><p>Python is a trendy language for machine learning (ML) projects. It is one of the easiest programming languages to learn. Not only does it offer simplicity, but it also has a wide range of libraries to perform various tasks, making it one of the most versatile programming languages. However, simplicity comes at a cost — it is not easy to write efficient code in Python and most public packages use C++ under the hood to overcome this challenge.</p><p>In this article, we will discuss six tools you can use to improve your code quality, and then we’ll review the main phases in the Python package analysis. Let’s dive in!</p><h3>Tools That Improve Code Quality</h3><p>There are many tools and libraries that help us improve our code quality. Some, like Mypy, require static typing while others can be run without any extra effort on your part. (Note: Stating the object type is not mandatory in Python. However, if you do so, bugs can be easily found.) In this article, we will cover the following packages:</p><ul><li><strong>Black </strong>— Black is a PEP 8 compliant opinionated formatter. Black reformats entire files in place. Style configuration options are deliberately limited and rarely added.</li><li><strong>Flake8</strong> — Flake8 is a wrapper for the following tools:<br> 1. PyFlakes — Pyflakes only examines the syntax tree of each file individually. Consequently, Pyflakes is more limited in the types of things it can check.<br>2. Pycodestyle — Pycodestyle is a tool to check your Python code against some of the style conventions in PEP 8. <br>3. Ned Batchelder’s McCabe script — This script checks cyclomatic complexity in Software Testing, which is a testing metric used for measuring the complexity of a software program.</li><li><strong>Mypy — </strong>Mypy is an optional static type checker for Python that aims to combine the benefits of dynamic (or “duck”) typing and static typing. Mypy forces you to write with types and find mistakes in your code that Python allows (like sending a dict instead of a list to a function).</li><li><strong>Pylint </strong>— Pylint analyzes your code without actually running it. It checks for errors, enforces a coding standard, looks for <a href="https://martinfowler.com/bliki/CodeSmell.html">code smells</a>, and can make suggestions about how the code could be refactored. Pylint can also infer actual values from your code using its internal code representation.</li><li><strong>Pytest —</strong> The Pytest framework makes it easy to write small, readable tests and can scale to support complex functional testing for applications and libraries.</li><li><strong>Coverage</strong> — Coverage is a tool for measuring code coverage of Python programs. It monitors your program, noting which parts of the code have been executed, then analyzes the source to identify code that could have been executed but was not.</li></ul><h3>Phases of Python Package Analysis</h3><p>Here are the main phases in the Python package analysis:</p><ol><li>Install packages</li><li>Configure packages</li><li>Write tests</li><li>Configure CI/CD steps for automating reports</li><li>Using the output to fix code</li></ol><p>Now, we’ll learn about each of the individual phases in more detail.</p><h4>Phase 1: Install packages</h4><p>If you are using Pip, just write:</p><pre>pip install black flake8 mypy pylint pytest coverage  </pre><p>If you are using Poetry, use the following line:</p><pre>poetry add black flake8 mypy pylint pytest coverage</pre><p>Note: Details about Poetry installation and configuration can be found in the Links Index at the end of this article.</p><h4>Phase 2: Configure packages</h4><p>Black doesn’t need a special configuration. However, if you want to see some details on the available configuration options, please view the Links Index at the end of this article.</p><p><strong>Flake8 configuration</strong></p><p>You can configure Flake8 with command line arguments (<a href="https://flake8.pycqa.org/en/latest/user/options.html#index-of-options">https://flake8.pycqa.org/en/latest/user/options.html#index-of-option) </a>or .flake8 file in the project root. My preferred options are here:</p><pre>[flake8]<br>max-line-length = 170<br>exclude = .venv,.git,__pycache__,.pytest_cache,build,dist,.history<br>max-complexity = 12</pre><p>Here are two important notes:</p><ol><li>Long line length is due to logging messages. Most of the time, Black enforces shorter line lengths.</li><li>Code complexity is measured for each function separately. Every time there is an if block in Python, code cyclomatic complexity increases by 1. So, max complexity of 12 means that we allow up to 12 conditions in each function.</li></ol><p><strong>Mypy configuration</strong></p><p>You can configure Mypy with command line arguments or mypy.ini in the project root. A sample mypy.ini is shown below:</p><pre>[mypy]<br>python_version = 3.9<br># the following config is really hard to implement <br># when returning complex structures (list of dicts with different types)<br>#warn_return_any = True <br>warn_unused_configs = True<br>ignore_missing_imports = True</pre><p>Here is an explanation of the above:</p><ul><li>Configure your Python version to mypy.</li><li>warn_return_any means you will be forced to avoid “Any” declaration in the code. “Any” contradicts mypy static typing, it is better not to use it. However, when dealing with large data structures and complex ones, like nested dicts and lists with multiple objects type, it is more readable to use “Any” than to declare everything.</li></ul><pre>Dict[str,Dict[str,Dict[str,List[str]] <br>versus <br>Dict[str,Any]</pre><p>Therefore, in most projects, I turn it off.</p><ul><li>warn_unused_configs — Report any config options that are unused by mypy. (This will help us catch typos when making changes to our config file).</li><li>ignore_missing_import — Not all packages support Mypy syntax, so this configuration helps us avoid third-party libraries missing type hints (this will generate errors that we can’t fix since they are in the third-party code). Also, this can be configured per file like the example below:</li></ul><pre># Global options:<br><br>[mypy]<br>warn_return_any = True<br>warn_unused_configs = True<br><br># Per-module options:<br><br>[mypy-mycode.foo.*]<br>disallow_untyped_defs = True<br><br>[mypy-mycode.bar]<br>warn_return_any = False<br><br>[mypy-somelibrary]<br>ignore_missing_imports = True</pre><p><strong>Pylint configuration</strong></p><p><a href="https://pypi.org/project/pylint/">Pylint</a> is a rich tool with many configurations. Here, I will show you some important options. Generating a configuration file can be done using the following command (remove “poetry run” if you don’t use Poetry):</p><pre> poetry run pylint --generate-rcfile | out-file -encoding utf8 .pylintrc</pre><p>Now, let’s go over the main configuration keys that you should know.</p><p>Under the <em>master</em> section:</p><ul><li><em>disable</em> — Some Pylint checks possess many false alarms, so this is the place to add all the false positive checks or Pylint checks that you would like to ignore.</li><li><em>fail-under</em> — <strong>This is the most important configuration in Pylint, in my opinion. </strong>Once you set up a scoring mechanism for your code, then each commit will calculate the score and if it is below the number defined, Pylint will fail the build. I use 8 as the lower limit for code quality. If you start using Pylint from the first line of code that you commit, I would recommend setting it to 9.</li></ul><p>Under the <em>report</em> section:</p><ul><li><em>evaluation</em> — This is the formula to calculate the score. The default setting here implies that errors are 5 times more important than other issues (e.g. warning, refactor, convention), but you can play with the formula as you wish to create a tailored score. Mind that if you change the formula from the default setting, the <em>fail-under</em> should be changed as well.</li></ul><p>Under the <em>format</em> section:</p><ul><li>max-line-length — This should be a number that matches the line length of Flake8.</li><li>max-module-lines - This is the maximum number of lines in the module.</li></ul><p>Under the <em>basic</em> section:</p><ul><li>good-names — Here, you can add all your short agreed variable names. For example, “df,f,i,k” and so on. This will tell Pylint not to warn you about those names.</li></ul><p>Under <em>design</em> section:</p><ul><li>Go over all the parameters and set your desired coding style.</li></ul><h4>Phase 3: Write tests</h4><p>It’s very important to learn how to write good tests. However, since writing tests depends on the repository and code you are using, it’s a bit difficult to provide relevant tips. Instead, I will direct you to the following link, where you can learn how to write small, readable tests that can scale to support complex functional testing for various applications and libraries.</p><ul><li><a href="https://docs.pytest.org/en/7.2.x/">pytest: helps you write better programs — pytest documentation</a></li></ul><p>once you finished writing tests, use the Coverage package to measure the coverage of your tests. Coverage.py is a tool for measuring code coverage of Python programs. It monitors your program, noting which parts of the code have been executed, then analyzes the source to identify code that could have been executed but was not. here is an example of coverage output.</p><pre>$ coverage report -m<br>Name                      Stmts   Miss  Cover   Missing<br>-------------------------------------------------------<br>my_program.py                20      4    80%   33-35, 39<br>my_other_module.py           56      6    89%   17-23<br>-------------------------------------------------------<br>TOTAL                        76     10    87%</pre><p>we can understand for each file, which lines we cover and which lines need more testing.</p><h4>Phase 4: Configure CI/CD steps for automating reports</h4><p>In order to configure CI/CD, you will need to copy the following lines to your CI file (.gitlab-ci.yml):</p><pre>stages:<br>  - static code analysis and tests<br><br>static_code_analysis:<br>  stage: static code analysis and tests<br>  script:<br>    - pip install poetry<br>    - poetry install<br>    - poetry run black . --check<br>    - poetry run flake8 --max-line-length=150<br>    - poetry run mypy project_path/<br>    - poetry run pylint project_path/<br>    - poetry run coverage run -m pytest -vs tests<br>    - poetry run coverage report -mya</pre><p>This step does the following:</p><ul><li>install packages</li><li>run static checks</li><li>run tests and coverage (assuming the tests are under tests folder)</li><li>issue reports</li></ul><p>If you are not using Poetry, use the following configuration (assuming that Pip installed all dependencies in the docker):</p><pre>stages:<br>  - static code analysis and tests<br><br>static_code_analysis:<br>  stage: static code analysis and tests<br>  script:<br>    - black . --check<br>    - flake8 --max-line-length=150<br>    - mypy project_path/<br>    - pylint project_path/<br>    - coverage run -m pytest -vs tests<br>    - coverage report -mya</pre><h4>Phase 5: Using the output to fix code</h4><p>When you are finished installing and configuring all packages, you will run X. Then, each tool will provide an output listing the errors and suggestions for correction of your code. At this point, you should go over the errors and warnings, check if they are valid (if not you can remove this check in the next run using the configuration files), and fix your code.</p><h3>Pyreverse example</h3><p>Pyreverse is a small tool that comes with Pylint and it is very useful for creating code diagrams. See: <a href="https://pylint.pycqa.org/en/latest/pyreverse.html">https://pylint.pycqa.org/en/latest/pyreverse.html</a> for details.</p><p>In many cases creating a single diagram depicting all classes in the project yields a rather unwieldy, giant diagram. While limiting the input path to a single package or module can already help greatly to narrow down the scope, the -c option provides another way to create a class diagram focusing on a single class and its collaborators. To extract the class diagram, simply write the following in your repo (point to the right class or folder):</p><pre>poetry run pyreverse -o png -s1 -c sklearn.linear_model._base.LinearRegression sklearn/linear_model</pre><p>Below is an example of the output that can be generated using Pyreverse for the linear regression class of sklearn:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/865/1*1miDchKLvvxWTnefq3qASg.png" /></figure><p>All class members and functions can be visualized for documentation of the code classes</p><h3>Links Index:</h3><ul><li>Poetry site — (<a href="https://python-poetry.org/docs/">Introduction | Documentation | Poetry — Python dependency management and packaging made easy</a>)</li><li>Black site— <a href="https://black.readthedocs.io/en/stable/">https://black.readthedocs.io/en/stable/</a></li><li>Flake8 site — <a href="https://flake8.pycqa.org/en/latest/">https://flake8.pycqa.org/en/latest/</a></li><li>Mypy site — <a href="https://mypy.readthedocs.io/en/stable/getting_started.html">https://mypy.readthedocs.io/en/stable/getting_started.html</a></li><li>pylint site — <a href="https://pypi.org/project/pylint/">https://pypi.org/project/pylint/</a></li><li>coverage site — <a href="https://coverage.readthedocs.io/en/6.5.0/">https://coverage.readthedocs.io/en/6.5.0/</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7348b1bb0289" width="1" height="1" alt=""><hr><p><a href="https://medium.com/syte-engineering/python-code-quality-checks-7348b1bb0289">Python Code Quality Checks</a> was originally published in <a href="https://medium.com/syte-engineering">syte_engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Green Trunk Approach]]></title>
            <link>https://medium.com/syte-engineering/the-green-trunk-approach-2c5aee87999e?source=rss----2fbf22bb84ad---4</link>
            <guid isPermaLink="false">https://medium.com/p/2c5aee87999e</guid>
            <category><![CDATA[green-trunk]]></category>
            <category><![CDATA[staging]]></category>
            <category><![CDATA[git]]></category>
            <category><![CDATA[deep-linking]]></category>
            <category><![CDATA[jira]]></category>
            <dc:creator><![CDATA[Stanislav Spiridonov]]></dc:creator>
            <pubDate>Thu, 15 Dec 2022 10:50:58 GMT</pubDate>
            <atom:updated>2022-12-15T10:50:58.277Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fI_4GBw3eWai-QbhbVHApA.jpeg" /></figure><p>In this article, I’m going to describe the green trunk approach which helps to increase the quality and speed of new features deployments. Before we dive in, let’s review some basics first.</p><h3>The Classic Staging Approach</h3><p>Let’s assume that we have a traveling circus group. Every performer has their own act (or program) and every performance needs to be practiced and, usually, slightly modified in each new location in order to be completed correctly and safely. Once the program is modified appropriately, the show can go on without any further changes for that specific location.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*07dTeOEsTLoPZjAcx2WhzQ.jpeg" /></figure><p>The same approach is used in IT — meaning, before actually deploying a new version of an application to production, a separate staging environment is used for testing purposes. Just like the traveling circus group, once the application is perfected in the staging environment, it can then be presented to the intended audience.</p><p>It’s the classic way of completing IT deployment cycles. Generally, the workflow looks like this: we have a master branch for production purposes and one or more staging branches for testing purposes. Every new task is done in a separate feature branch, tested locally and merged via pull request into a staging branch. During a delivery window, the code from staging should be tested and deployed to production by merging the staging branch into the master branch.</p><h3>Drawbacks of Single Staging Environment</h3><p>However, the classic staging approach has weaknesses. For example, if there are big changes that will have a broad impact (e.g. switching one of the core libraries), the staging could be frozen until everything is thoroughly tested and all issues are resolved.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IJkWo3cAPUs98UwM5SYtwg.jpeg" /></figure><p>If this happens, we have two choices:</p><h4>Option 1: Revert this task from staging</h4><p>If we choose this option, we must complete the following three steps before we can continue:</p><ol><li>solve the issue</li><li>merge it back to staging</li><li>return to testing again</li></ol><p>The consequences of reverting a task from staging include:</p><ul><li>It will leave a trail in the history log.</li><li>We will not have an environment to test all issues except locally (until the issue is resolved and merged back into staging again).</li><li>If we have a lot of subsequent issues, this process can involve many cycles and take a lot of time.</li></ul><h4>Option 2: Freeze the deliveries and stabilize the staging environment</h4><p>If we choose the second option, it means we can’t deploy other smaller changes like updating title text or the style of specific elements before all issues in staging are resolved.</p><p>This could have the following consequences:</p><ul><li>Freezing deliveries could heavily decrease the effectiveness of the project.</li><li>The fact that every deployment contains a number of different features makes the revert process a bit tricky. Ultimately, one bug in one feature could ruin a lot of others when delivered in a single, inclusive update. This is especially true if we have features dependencies and we can not revert only one of them — only the full bulk of features.</li></ul><h3>How Can We Avoid These Drawbacks</h3><p>With the green trunk approach, we can avoid the problems above because every feature has its own environment which can be tested separately. Each branch is actually the production plus feature commits, like a sandbox for each specific feature.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cohC7W0lu7fOlpC2eVsqaQ.jpeg" /></figure><p>Now, let’s discuss the main steps that should be done in order to achieve a green trunk.</p><h4>Dynamic environments</h4><p>The first thing we have to do in order to support green trunk methodology is to configure our CI/CD in order to be able to deploy every feature branch into a specific environment. It could be a specific docker image for BE parts, different paths for web applications, specific installer programs, etc.</p><p>Now, assume we already have production and staging environments. What we have to do is build a staging-like environment for a dynamic path (usually the branch name since it’s unique). There are several options here:</p><ul><li>For web applications, we could just deploy it for a specific path.</li><li>For BE staff, we can build a specific docker image.</li><li>For databases, we can clone some part of the data (probably with some post-processing like anonymous data, etc).</li></ul><h4>Deep links</h4><p>When we have only production and staging environments, things can be hardcoded (e.g. production and staging URLs for web applications, different database addresses, etc).</p><p>If we are going to support a bunch of environments at the same time, we have to find a way to provide a specific environment parameter in order to switch part of our application to work with a specific branch environment. The solution here is deep links. For example, a deep link could be a query parameter for web apps and API, name of docker image to use, etc. For safety reasons it would be good to add some validation rules for such parameters so it will allow only specific addresses.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eOwTRwK2TrsfvIPg92YBrA.jpeg" /></figure><p>Deep links are a great improvement in terms of performance and support since they allow developers, QA members, product managers, etc to share specific environment configurations by just sending one link. The same links or mechanisms could also be used by auto testing tools.</p><h4>Issues tracker integration</h4><p>After we have an ability to deploy a dedicated environment and build one link, we can use it in order to integrate those links with issue tracking systems like JIRA, Trello, Monday, etc. The only thing we need to add is one more step in CI/CD pipelines in order to attach a link or comment with a link to a specific environment. In JIRA, for example, the web links section can be used. This allows the developer to push new changes into the repository and then a new version will be published and QA members (and anybody else related) can use this link to see the changed version without any need to run it locally.</p><h4>Git-flow</h4><p>In order to use green trunk at full power, we have to switch git flow strategy to Fast-Forward master branch. Doing so enables us to be sure that there won’t be any changes between the branch environment that we tested and that the same code base (without any changes related to the merge process) will be applied to the master branch and deployed to production.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hD4ITpb1LDKqV51dO-ODxg.jpeg" /></figure><p>Note: This still would not be a panacea since the code itself could still contain some production / non production scripts, which should be minimized in order to increase stability. Examples could include minification of production scripts, omitting debug and asserts, etc.</p><h3>Green Trunk Development Flow</h3><p>Now that we’ve discussed all the necessary steps required to support the green trunk approach, let’s take a moment to consider green-trunk flow from a developer’s point of view.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ikTvxSsVz3YwPsRSQDnnVg.jpeg" /></figure><p>When a developer starts working on a task, the work starts from the creation of a branch for the task. After development is done, the pull / merge request to master should be created. After passing code review, the task will go to QA and all necessary links should already be added to the task in the issue tracker. After QA has finished testing with the provided links, the pull / merge request can be merged and a new version including this new feature / bugfix will be deployed to production. Finally, all other branches should be rebased (and conflicts should be resolved if any) onto a new master branch in order to get the new changes. This will enable every branch to be exactly the same as current production plus the changes required for specific features.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2c5aee87999e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/syte-engineering/the-green-trunk-approach-2c5aee87999e">The Green Trunk Approach</a> was originally published in <a href="https://medium.com/syte-engineering">syte_engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[OCR for visual similarity]]></title>
            <link>https://medium.com/syte-engineering/ocr-for-visual-similarity-6dc09545d7d6?source=rss----2fbf22bb84ad---4</link>
            <guid isPermaLink="false">https://medium.com/p/6dc09545d7d6</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[optical-character-recogn]]></category>
            <category><![CDATA[visual-modeling]]></category>
            <category><![CDATA[similarity-search]]></category>
            <dc:creator><![CDATA[Yair Vagshal]]></dc:creator>
            <pubDate>Wed, 14 Sep 2022 14:06:17 GMT</pubDate>
            <atom:updated>2022-09-19T19:01:27.752Z</atom:updated>
            <content:encoded><![CDATA[<h4>Optical Character Recognition</h4><p>An article by <a href="https://www.linkedin.com/in/maorn/">Maor Nissan</a>, <a href="https://www.linkedin.com/in/yair-vagshal-34458221b/">Yair Vagshal</a></p><p>Visual similarity is a core component of S<a href="https://www.syte.ai/">yte</a>’s product offerings. While this product can be used on its own, we use Visual Similarity to power different products as well. The challenge with Visual Similarity is that is subjective. Some view similarity as anything that shares visual attributes, such as color, sleeve length, or material.</p><p>One of the challenges with Visual Similarity can be explained as follows: someone is trying to decorate a room with items that are all of the same style. Style is not always easy to express in words- for example, what is Scandinavian design in furniture and home decor?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/256/0*ZefZuRsHDLz9i770" /></figure><p>According to the definition, Scandinavian design is “Characterized by <strong>a minimal, clean approach that seeks to combine functionality with beauty</strong>. Its focus is on simple lines and light spaces devoid of clutter.”</p><p>The definition of the Scandinavian style is vague and doesn’t possess concrete attributes and values of the items that are considered Scandinavian Style. The items can vary in color and material but are still considered similar by the consumer and the industry.</p><p>Another challenge in Visual Similarity is when we are trying to provide results for similar themes, such as the same sports team, cinematic universe, or cast of characters. For example, Liverpool F.C. shirts may look visually similar to Manchester United shirts in their attributes (category, type, sleeve length), but the chances that Liverpool fans will want Manchester united shirt is low.</p><p>Based on these observations, we wanted to identify the text hidden in images and use it as input for our similarity engine as one of the factors that influence our distance measure. If we know that the item contains a Star Wars attribute or theme, we could match other star wars items.</p><h3>Optical Character Recognition</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OdICm0hw9NkL88bvLq7FrA.jpeg" /><figcaption>Image from Wikipedia</figcaption></figure><p>Optical Character Recognition is a model used to extract text from images. OCR models are used on various images to extract text from scanned documents, license plates, images with handwritten text, multilingual text, and more.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*nIGs66BfrWQuEqwD.jpg" /><figcaption>Figure 1: Example of a multilingual text extraction (Image from PaddleOCR’s website)</figcaption></figure><p>Our project’s task was to extract text from clothing products, such as shirts, hats, bags, shoes, etc. The text would be used to get information about the product and then recommend similar items. For instance, a shirt with the text ‘Nirvana’ written on it could indicate that it’s a Band shirt in the rock category, hence more rock band shirts could be suggested.</p><p>In this particular task, there are many types of texts that the OCR model needs to be able to extract. Each image could have text with different fonts, handwriting style, background noise, curves, blurriness, low resolution, etc. The model needs to be robust in order to handle all of these cases, while still inferring results fast enough to be practically usable.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/775/1*orDb3uAeHiUkrSSjr_JzLg.png" /><figcaption>Figure 2: Examples of the OCR model’s outcome with different fonts</figcaption></figure><p>In the following article, we detail the project and the process of testing and evaluating several OCR models. Moreover, we discuss several image-processing and text-processing that can help improve the results OCR model results.</p><p>Throughout the testing process, we used the following pipeline:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RDhcblU0yc9Hl_LeECy9tQ.png" /><figcaption>Figure 3: The testing pipeline</figcaption></figure><h3><strong>Dataset</strong></h3><p>The dataset we used for testing the OCR pipeline has images of shirts, bags, shoes, and hats with different themes, such as music genres, comic characters, cinematic universes, and sports teams.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_Wm6MKU12zjBDLJVO3mi-Q.png" /><figcaption>figure 4: Examples from the dataset</figcaption></figure><h3><strong>Metrics</strong></h3><p>In order to evaluate the output of the pipeline, we use three types of metrics:</p><ol><li><strong>CER:</strong> Character Error Rate. <a href="https://huggingface.co/spaces/evaluate-metric/cer">link to source</a></li><li><strong>WER:</strong> Word Error Rate. <a href="https://huggingface.co/spaces/evaluate-metric/wer">link to source</a></li><li><strong>BLEU:</strong> Bilingual Evaluation Understudy. <a href="https://huggingface.co/spaces/evaluate-metric/bleu">link to source</a></li></ol><p>CER &amp; WER both use <a href="https://en.wikipedia.org/wiki/Levenshtein_distance#:~:text=Informally%2C%20the%20Levenshtein%20distance%20between,considered%20this%20distance%20in%201965.">Levenshtein distance</a>, which measures the difference (error) between two string sequences.</p><p><strong>The formula of CER is:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/637/1*-7oEBldOMlirI7Xp6iy4VQ.png" /></figure><p><strong>Example of Character Error Rate (CER):</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/744/1*rzbQN0W9F9bpk4DZJMeuIg.png" /></figure><p>CER can be misleading — good results of the character’s output might not be enough to evaluate the model’s performance, and we might have a low error rate despite there being an error in each one of the words. To overcome this problem, we also use WER, which has a similar idea but with the word’s output.</p><p><strong>The formula of WER is:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/629/1*hJovdE2GaiY0lhJ-mBemTg.png" /></figure><p><strong>Example of Word Error Rate (WER):</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/741/1*9O5xpHpi8kLc1FuuHU5plw.png" /></figure><p>While these two metrics cover both characters and word output, it is still not enough. For example, if the model extracts all the words correctly but in the wrong order, WER won’t give us good results.</p><p>Bleu gives its score based on consecutive correct words (n-grams):</p><ul><li>Bleu1 gives a 100% success rate when all the words are correct (no matter their order)</li><li>Bleu2 gives a 100% success rate when all the possible consecutive two words are correct</li><li>Bleu3 gives a 100% success rate when all the possible consecutive three words are correct, etc.</li><li>Bleu is the average of all the scores (the number of scores is a parameter)</li></ul><p><strong>Example of BLEU2:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fqS3FwJimXB_LMIVHnlkew.png" /><figcaption>Figure 5: Example of BLEU2- Looking for all of the 2 consecutive words</figcaption></figure><h3><strong>Models</strong></h3><p>There are many types of OCR models, and they use many types of deep networks: CNN, RNN, Transformers (Vision model, Language models), Segmentation models, etc.</p><p>We tested the following models: EasyOCR, PaddleOCR, CRNN, MASTER, NRTR, RobustScanner, SAR, SATRAN, SegOCR, CRNN-STN, and ABINet. Out of these models, we focused on the two that gave the best results:</p><ol><li><strong>ABINet. </strong><a href="https://arxiv.org/pdf/2103.06495.pdf">link to article</a></li><li><strong>PaddleOCR, PP-OCRv3. </strong><a href="https://arxiv.org/pdf/2206.03001v1.pdf">link to article</a></li></ol><h4><strong>ABINet</strong></h4><p>ABINet uses a Vision model and a language model to recognize text in the wild, which are trained in an end-to-end way. ABINet Pipeline:</p><ol><li>Vision Model- ResNet + transformer</li><li>Language model- Transformer</li><li>ABINet works iteratively and uses both models to increase the probability of the outcome.</li><li>Theoretically, the language model can be replaced by other model languages (Transformers)</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/0*iHgE4XO9x9ZzXqoA" /><figcaption>Figure 6: ABINet’s structure (Image from the model’s article)</figcaption></figure><h4><strong>PaddleOCR, PP-OCRv3</strong></h4><p>PaddleOCR is an OCR framework that provides multilingual practical OCR. PaddleOCR Pipeline:</p><ol><li>Text Detection (DB): Probability map of the location of the characters</li><li>Detection Boxes Rectify: Corrects the text’s direction</li><li>Text Recognition (SVTR): a Vision Transformer that outputs the text</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/875/0*sfKn2ecep9vXwRZz" /><figcaption>Figure 7: PaddleOCR’s pipeline (Image from the model’s article)</figcaption></figure><h3>Models’ Error Analysis</h3><p>We performed error analysis for the models in order to understand their sensitivity to different types of texts. To accomplish this, we used the repository <a href="https://github.com/Belval/TextRecognitionDataGenerator">TextRecognitionDataGenerator</a> and we were able to create many types of texts for the analysis. The conclusions from the analysis helped improve the results of the pipeline.</p><p>For example, the analysis revealed that PaddleOCR is highly sensitive to background noise and texts at certain angles. In the case of angled texts, it seems that the ‘Detection Boxes Rectify’ could be improved by training with more examples of these certain angles. And in the case of background noise, we found that removing background noise during the preprocessing stage improves performance.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*np_xwizU5Isz1lXH635IqQ.png" /><figcaption>Figure 8: Example of background noise’s effect on the model’s output</figcaption></figure><h3><strong>Pre-Processing the Image</strong></h3><h4><strong>The goals of pre-processing the images were:</strong></h4><ol><li>Enhancing the difference between the text and background (clearer borderlines).</li></ol><p>To achieve this goal we tried the following methods:</p><ul><li>Stretch the histogram of the pixels’ values to make a more significant difference between areas in the image (<strong>Histogram Equalization, CLAHE).</strong></li><li>Edge detection, in order to find the borderline of the characters <strong>(Sobel, Canny)</strong></li><li>Image Sharpening and Image threshold, in order to enhance the borderline of the characters. <strong>(Global, Adaptive Mean, Adaptive Gaussian)</strong></li></ul><p>2. Make the text and the background clearer (Remove noise from both while keeping the borderline prominent).</p><p>To achieve this goal we tried the following methods:</p><ul><li>Smoothing Images<strong> (Low-pass Filters- Gaussian, Median, Mean, Non-local Means)</strong></li><li>Smoothing Images while keeping the edges<strong> (Bilateral Filter)</strong></li></ul><p>These are some of the results of the models with image pre-processing:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/770/1*jn7ilavZqdcAszroVD6NkQ.png" /></figure><p>The most effective pre-processes involved removing noise, whereas the other methods reduced the outcomes’ results. The best results came from Bilateral Filter, which is a low-pass filter that takes into consideration the distance between the pixels. When the distance is further the effect of the filter becomes smaller, and therefore it won’t smooth edges but will smooth noise.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/0*7SP-kgkuc16U7h0a" /><figcaption>Figure 8: Bilateral Filter</figcaption></figure><h3><strong>Final results</strong></h3><p>These are the final results of the best models’ configurations:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/801/1*fVaFYWrb1pXNckygtf7okw.png" /></figure><ul><li>Using Bilateral Filter with ABINet increased the performance in the character’s outcome (CER) by ~10.5% and in the word’s outcome (WER) by ~28%</li><li>When looking at the word’s outcome (WER, BLEU2, BLEU3, BLEU4), PaddleOCR model got better results than ABINet</li><li>In general ABINet with Bilateral Filter gets better results than PaddleOCR with Bilateral Filter, except for BLEU3 and BLEU4. Even though PaddleOCR is much faster than ABINet, which makes it more usable.</li><li><strong>As a result of the above, the chosen pipeline for the project is PaddleOCR with Bilateral Filter.</strong></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/988/1*fxOBK1u_ACRXC9PNWhew6A.png" /><figcaption>Figure 9: Examples of PaddleOCR with Bilateral Filter</figcaption></figure><p>In conclusion, we will use the PaddleOCR model with Bilateral Filter as an input for our visual similarity search. This solution will enable us to provide similar suggestions for themes, cinematic universes, brands, sports teams, and any other topics that are described in the text.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6dc09545d7d6" width="1" height="1" alt=""><hr><p><a href="https://medium.com/syte-engineering/ocr-for-visual-similarity-6dc09545d7d6">OCR for visual similarity</a> was originally published in <a href="https://medium.com/syte-engineering">syte_engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Case Study: Using Learning to Rank to Improve Item Ranking With Elasticsearch and RankLib]]></title>
            <link>https://medium.com/syte-engineering/case-study-using-learning-to-rank-to-improve-item-ranking-with-elasticsearch-and-ranklib-ea47fd5072e6?source=rss----2fbf22bb84ad---4</link>
            <guid isPermaLink="false">https://medium.com/p/ea47fd5072e6</guid>
            <category><![CDATA[product-recommendations]]></category>
            <category><![CDATA[learning-to-rank]]></category>
            <category><![CDATA[search-engines]]></category>
            <category><![CDATA[elasticsearch]]></category>
            <dc:creator><![CDATA[Einav Tamam]]></dc:creator>
            <pubDate>Tue, 05 Apr 2022 12:36:34 GMT</pubDate>
            <atom:updated>2022-04-06T17:33:13.976Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FjvYS6oSsHleJuJZ_8uObw.jpeg" /></figure><h3>Case Study: Using Machine Learning to Improve Item Ranking With Elasticsearch and RankLib</h3><p>As shoppers — and as the people building recommendation systems — we always want to see (or to show) the most relevant items on top in any given context. Many eCommerce brands put too much effort into bringing users to their website and not enough effort into improving the ranking of their search results and product recommendations to ensure those users find the best matches for them immediately during their shopping session.</p><p>But what does relevance mean, exactly? Ultimately, we’re referring to items that a user will click on and go on to purchase. But these criteria are not fixed, nor are they easy to measure.</p><p>In this case study, I’ll share how we used machine learning models to rank items within a text search engine (also known as “query document ranking”).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uPv6dcllrA8bQw83UNOQtw.png" /></figure><p>So, let’s start with the million-dollar question that we were facing: What makes an item clickable?</p><ul><li>The item should match the search query</li><li>The item should be similar to products the user clicked on in the past</li><li>The item should align with the user’s context and journey (i.e. location, time of day, season, gender of items searched for)</li><li>The item should align with the user profile (i.e. existing/new customer, has purchased/has not purchased)</li></ul><p>To begin with, we decided to take just the features of the items themselves into consideration.</p><h3>What Is Learning to Rank?</h3><p>Learning to Rank (LTR) is a combination of supervised and semi-supervised techniques of predicting product relevance.</p><p>With this type of ranking model, we needed to:</p><ul><li>Create a judgment list — rank each item’s relevance in each search query</li><li>Extract the item features — can be textual (e.g. title, description), numeric (e.g. price), or string (e.g. color).</li></ul><h4>Creating a Judgment List</h4><p>Labeling data is the most challenging part of many ML models — more specifically, labeling it correctly. There are a few ways to create a judgment list, but in our case, we focused on two of them: implicit feedback and an expert panel.</p><p>An expert panel is the most accurate way to label data, but it’s more expensive and time-consuming, plus, different experts can have different opinions/scores.</p><p>Implicit feedback is based on actions from the user journey — like clicks, add to cart, or even purchases of items. With this approach, we assumed that clicks signal relevance, but this is not always the case.</p><p>The major advantage here is that you can gain new insights from existing data — it’s a simple and fast way to create a label.</p><p>Since we were working on improving the relevance of product rankings on eCommerce sites, the best practice was to use implicit feedback with the COEC (Click-Over-Expected-Clicks) model — a linear regression that predicts the relevance score of an item based on the impressions per position. This model aims to predict the click-through rate (CTR) of an item if it was the only result shown.</p><p>The training set for the COEC model looks like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/506/1*YaqS4ucR01yxiXbgREGAtw.png" /><figcaption>Impressions per position and clicks per item per query</figcaption></figure><p>The new rank is calculated by the predicted clicks — the ratio between the expected and the actual — as you can see here:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/589/1*3yV7tAqqsTdBOPqRxGrw8g.png" /></figure><p>The new rank reflects the highest ratio as first place and continues in descending order.</p><h4>Extracting Item Features</h4><p>To train the model, we needed to log feature values. This is a major component of the model. As users search, we logged feature values from the feature set in Elasticsearch so that we could start training.</p><p>In order to perform feature extraction with the Elasticsearch plug-in, we first created the feature set in Elasticsearch. This requires two components:</p><ul><li>Validation: sanity check that the features work as expected on a specific index</li><li>Feature set: list of features where a feature can return the similarity score between a query and document or the exact value of a numeric feature</li></ul><pre>POST {ES_URL}/_ltr/_featureset/{feature_set_name}<br>{<br> &quot;validation&quot;: {<br>  &quot;params&quot;: {<br>   &quot;keywords&quot;: &quot;red dress&quot;<br>  },<br>  &quot;index&quot;: &quot;index_name&quot;<br> },<br> &quot;featureset&quot;: {<br>  &quot;features&quot;: [{<br>    &quot;name&quot;: &quot;1&quot;,<br>    &quot;params&quot;: [<br>     &quot;keywords&quot;<br>    ],<br>    &quot;template&quot;: {<br>     &quot;multi_match&quot;: {<br>      &quot;type&quot;: &quot;most_fields&quot;,<br>      &quot;query&quot;: &quot;{{keywords}}&quot;,<br>      &quot;fields&quot;: [<br>       &quot;title.*&quot;<br>      ]<br>     }<br>    }<br>   },<br>   {<br>    &quot;name&quot;: &quot;2&quot;,<br>    &quot;params&quot;: [<br>     &quot;keywords&quot;<br>    ],<br>    &quot;template&quot;: {<br>     &quot;multi_match&quot;: {<br>      &quot;type&quot;: &quot;most_fields&quot;,<br>      &quot;query&quot;: &quot;{{keywords}}&quot;,<br>      &quot;fields&quot;: [<br>       &quot;description.*&quot;<br>      ]<br>     }<br>    }<br>   },<br>   {<br>    &quot;name&quot;: &quot;3&quot;,<br>    &quot;params&quot;: [],<br>    &quot;template&quot;: {<br>     &quot;function_score&quot;: {<br>      &quot;field_value_factor&quot;: {<br>       &quot;field&quot;: &quot;Price&quot;,<br>       &quot;modifier&quot;: &quot;none&quot;,<br>       &quot;missing&quot;: 0<br>      }<br>     }<br>    }<br>   }<br>  ]<br> }<br>}</pre><p>Once we had a feature set, we were able to extract the values of each item as follows:</p><pre>{<br>    &quot;query&quot;: {<br>        &quot;bool&quot;: {<br>            &quot;filter&quot;: [<br>                {<br>                    {&quot;sku.keyword&quot;:[&quot;111&quot;,&quot;222&quot;,&quot;333&quot;,&quot;444&quot;]<br>                },<br>                {<br>                    &quot;sltr&quot;: {<br>                        &quot;_name&quot;: &quot;logged_featureset&quot;,<br>                        &quot;featureset&quot;: {feature_set_name},<br>                        &quot;params&quot;: {<br>                            &quot;keywords&quot;: &quot;red dress&quot;<br>                        }<br>                    }<br>                }<br>            ]<br>        }<br>    },<br>    &quot;ext&quot;: {<br>        &quot;ltr_log&quot;: {<br>            &quot;log_specs&quot;: {<br>                &quot;name&quot;: &quot;log_entry&quot;,<br>                &quot;named_query&quot;: &quot;logged_featureset&quot;,<br>                &quot;missing_as_zero&quot;:&quot;true&quot;<br>            }<br>        }<br>    }<br>}</pre><p>And then we saved the results in a database and used it for training.</p><h3>Training a Ranking Model</h3><p>Now, we finally have a training set of items and queries, which includes a key (query, SKU), features (similarity score or numeric values), and a label (relevance score).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/634/1*_ubkB1z4xMZYcSE7t0O5Jw.png" /></figure><p>For training, we used the RankLib Java package in Python script. RankLib is a library of Learning to Rank algorithms. Currently, eight popular algorithms have been implemented:</p><ul><li>RankNet [1]</li><li>RankBoost [2]</li><li>AdaRank [3]</li><li>Coordinate Ascent [4]</li><li>LambdaMART [5]</li><li>MART (Multiple Additive Regression Trees, AKA Gradient boosted regression tree) [6]</li><li>ListNet [7]</li><li>Random Forests [8]</li></ul><p>Below is the CMD command for training a model with RankLib. We defined the model number that we wanted to train with and the filenames of the training data source and model output. We set 20% of the training for validation test (TVS) and selected the optimization metric (NDCG@10).</p><p>The MART algorithm, model number 6, received the highest evaluation metrics on our training and validation sets.</p><pre>cmd = f&quot;java -jar RankLibPlus-0.1.0.jar -ranker 6  -train &quot;train_data.txt&quot; -save &quot;model_output.txt&quot; -tvs 0.2 -metric2t NDCG@10 -frate 1.0&quot;<br>Logger.logger.info(&quot;**********************************************&quot;)<br>Logger.logger.info(&quot;**********************************************&quot;)<br>Logger.logger.info(&quot;Running %s&quot; % cmd)<br>os.system(cmd)</pre><p>You can find more model parameters at this <a href="https://sourceforge.net/p/lemur/wiki/RankLib%20How%20to%20use/">link</a>.</p><h3>Evaluating Model Accuracy</h3><p>Great, we created a model! But how accurate would it be? Before going live with A/B testing, we evaluated our model using offline metrics such as NDCG. In the online tests, the metrics will be more business-wise, such as CTR, conversion rate (CVR), or average revenue per user (ARPU).</p><p>The <a href="https://towardsdatascience.com/evaluate-your-recommendation-engine-using-ndcg-759a851452d1">Normalized Discounted Cumulative Gain</a> (NDCG@N) metric is a well-known metric that measures the effectiveness of web search engine algorithms. It sums up the relevance score and divides it by its position log. The more relevant the item, the greater the penalty if it appears in lower positions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/620/1*G6L0b1sEJcSmarjy3owKUw.png" /><figcaption>DCG Score Example</figcaption></figure><h3>Uploading an LTR Model to Elasticsearch</h3><p>Once the model and the offline metric scores were good enough, it’s time to deploy it and test it online.</p><p>You can save the trained RankLib model in Elasticsearch by sending a post request of createmodel to Elasticsearch.</p><pre>import requests</pre><pre>model_payload = {<br>    &quot;model&quot;: {<br>        &quot;name&quot;: &quot;my_model&quot;,<br>        &quot;model&quot;: {<br>            &quot;type&quot;: &quot;model/ranklib&quot;,<br>            &quot;definition&quot;: {<br>            }<br>        }<br>    }<br>}</pre><pre>with open(&quot;my_model.txt&quot;) as model_filename:<br>    model_content = model_filename.read()<br>    model_payload[&#39;model&#39;][&#39;model&#39;][&#39;definition&#39;] = model_content</pre><pre>path = f&quot;{ES_URL}_ltr/_featureset/{feature_set_name}/_createmodel&quot;<br>resp = requests.post(path, data=json.dumps(model_payload), headers={&#39;Content-Type&#39;: &#39;application/json&#39;})</pre><h3>Searching With LTR</h3><p>Select a window_size for the rescore, here the window_size is 1,000, which means it re-scores the top 1,000 results for each query.</p><pre>POST {ES_URL}/{index_name}/_search<br>{<br>    &quot;query&quot;: {<br>        &quot;match&quot;: {<br>            &quot;_all&quot;: &quot;red dress&quot;<br>        }<br>    },<br>    &quot;rescore&quot;: {<br>        &quot;window_size&quot;: 1000,<br>        &quot;query&quot;: {<br>            &quot;rescore_query&quot;: {<br>                &quot;sltr&quot;: {<br>                    &quot;params&quot;: {<br>                        &quot;keywords&quot;: &quot;red dress&quot;<br>                    },<br>                    &quot;model&quot;: &quot;my_model&quot;<br>                }<br>            }<br>        }<br>    }<br>}</pre><h3>Conclusion</h3><p>I hope you enjoyed this look into our journey with Learning to Rank and feel more confident now about how to use the RankLib library for training a model and Elasticsearch for improving item ranking in real-time using the NDCG score and rank by relevance. This was my first blog post, and I’d love to hear from you in the comments about any topics you’d like to read about next!</p><h3>References</h3><ul><li><a href="https://elasticsearch-learning-to-rank.readthedocs.io/en/latest/">Elasticsearch Learning to Rank: the documentation - Elasticsearch Learning to Rank documentation</a></li><li><a href="https://sourceforge.net/p/lemur/wiki/RankLib/">The Lemur Project</a></li></ul><ul><li><a href="https://www.youtube.com/watch?v=6BGCn3h59nA&amp;ab_channel=Databricks">Applied Machine Learning for Ranking Products in an Ecommerce Setting Arnoud de Munnik Wehkamp Jerry</a></li><li><a href="https://towardsdatascience.com/evaluate-your-recommendation-engine-using-ndcg-759a851452d1">Evaluate your Recommendation Engine using NDCG</a></li><li><a href="https://www.youtube.com/watch?v=TG7aNLgzIcM&amp;ab_channel=Luminis">Learning to rank search results — Byron Voorbach &amp; Jettro Coenradie [DevCon 2018]</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ea47fd5072e6" width="1" height="1" alt=""><hr><p><a href="https://medium.com/syte-engineering/case-study-using-learning-to-rank-to-improve-item-ranking-with-elasticsearch-and-ranklib-ea47fd5072e6">Case Study: Using Learning to Rank to Improve Item Ranking With Elasticsearch and RankLib</a> was originally published in <a href="https://medium.com/syte-engineering">syte_engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>