<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[SMG Real Estate Engineering Blog - Medium]]></title>
        <description><![CDATA[SMG | Homegate | ImmoScout24 Engineering is happy to share ideas and technologies used in daily work. Our goal is to provide a great experience in offering and finding a new home. - Medium]]></description>
        <link>https://medium.com/smg-real-estate?source=rss----2186e5b9bd8f---4</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>SMG Real Estate Engineering Blog - Medium</title>
            <link>https://medium.com/smg-real-estate?source=rss----2186e5b9bd8f---4</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 01:03:40 GMT</lastBuildDate>
        <atom:link href="https://medium.com/feed/smg-real-estate" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Build a low-cost, low-latency, low-energy AI Translator — A DeepL alternative that works]]></title>
            <link>https://medium.com/smg-real-estate/build-a-low-cost-low-latency-low-energy-ai-translator-a-deepl-alternative-that-works-19b903d00845?source=rss----2186e5b9bd8f---4</link>
            <guid isPermaLink="false">https://medium.com/p/19b903d00845</guid>
            <category><![CDATA[translation]]></category>
            <category><![CDATA[google-translate]]></category>
            <category><![CDATA[machine-translation]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[deepl]]></category>
            <dc:creator><![CDATA[fm.mateo]]></dc:creator>
            <pubDate>Thu, 07 Aug 2025 12:57:31 GMT</pubDate>
            <atom:updated>2025-08-07T12:57:31.755Z</atom:updated>
            <content:encoded><![CDATA[<h3>Build a low-cost, low-latency, low-energy AI Translator: a DeepL alternative that works</h3><h4>A step-by-step guide to deploying your own AI-powered translation API at scale, and get rid of DeepL, Google Translate, Amazon Translate, etc.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*l7ogh_UBGj9zLMTxa5BX3w.png" /></figure><h3>Introduction and motivation</h3><p>At Swiss Marketplace Group (SMG), we operate multiple online platforms used daily by millions of people across Switzerland. One of them, Ricardo, is a second-hand marketplace where buyers and sellers interact in four different languages: French, German, Italian, and English.</p><p>To deliver a seamless experience across languages, we had been relying on <strong>DeepL API</strong>, one of the most accurate translation APIs on the market. And while DeepL served us well in terms of quality and response time, it came with significant downsides:</p><ul><li>A monthly cost of over<strong> 7,000 CHF</strong></li><li>A <strong>black-box approach</strong>, offering no insight into how translations were produced</li><li><strong>Data privacy risks</strong></li></ul><p>As usage grew, so did the bill, and the urgency to reduce it.</p><p>Around that time, generative AI had made enormous improvements, and open-source large language models (LLMs) had become more accessible than ever. This sparked a question:</p><blockquote><strong><em>Can we replace DeepL API with a self-hosted open-source LLM, without sacrificing translation quality, speed, and user experience?</em></strong></blockquote><p>The answer turned out to be yes. Over the course of my master thesis, I built a production-grade, GPU-accelerated translation service using modern LLM tooling, hosted entirely on Google Cloud Platform (GCP), and fully integrated into Ricardo’s infrastructure. The result:</p><ul><li><strong>Over 85% cost savings</strong></li><li><strong>Sub-3-second latency</strong></li><li><strong>No impact on conversion rate of buyers</strong></li></ul><p>And everything is reproducible, transparent, and extensible.</p><p>In this article, I’ll walk you through exactly how I built it, from selecting the right model, optimizing inference time, and wrapping it in an API, to deploying and monitoring it in production. My goal is simple: <strong>enable you to do the same</strong>.</p><p>Whether you’re a backend developer, a ML engineer, or just AI-curious, this guide will show you how to turn open-source LLMs into real-world, cost-efficient infrastructure.</p><h3>DeepL alternatives and model benchmarking</h3><p>Once we made the decision to move away from DeepL, the next step was to figure out what could realistically replace it. That meant surveying the landscape of translation solutions, from commercial APIs to open-source solutions, and running a thorough benchmark to find the best fit for our use case.</p><h4>Commercial APIs: a costly choice at scale</h4><p>I started by evaluating some of the major commercial alternatives: Google Translate, Amazon Translate, Microsoft Translator and others. These services offer decent translation capabilities and easy-to-use APIs, but the quality simply didn’t match DeepL, especially for other languages than English, like French, German, or Italian. The translations often lacked nuance, handled HTML formatting poorly, or failed to interpret user-generated content correctly. And even if slightly cheaper than DeepL, they were still costly at scale. In the end, none of them provided enough of an edge to justify a switch.</p><h4>Open-source systems: promising, but underwhelming</h4><p>Next, I explored open-source projects like LibreTranslate, Apertium, and OpenNMT. These are fully self-hostable and appealing from a privacy and cost perspective. However, they turned out to be too limited in practice. Most had outdated models, poor multilingual coverage, or unacceptable latency. Some couldn’t handle non-standard input like emojis, bullet points, or HTML tags, all of which are common on Ricardo, where users write their own listings. The performance gap with DeepL was simply too wide.</p><h4>Shifting gears: benchmarking modern AI models</h4><p>It became clear that we needed to look beyond traditional machine translation systems and focus on open-source AI models, which had made incredible progress thanks to projects like <a href="https://huggingface.co/">Hugging Face</a>. The goal was to find a model that was both accurate and fast enough to run inference on a single small GPU.</p><p>I shortlisted models like Facebook’s <a href="https://arxiv.org/pdf/2207.04672">NLLB-200</a>, <a href="https://arxiv.org/pdf/2001.08210">mBART</a>, Google’s <a href="https://jmlr.org/papers/volume21/20-074/20-074.pdf">T5</a> variants, the <a href="https://arxiv.org/pdf/2212.01936">Helsinki-NLP opus-mt</a> family, and finally, a newer contender: <a href="https://arxiv.org/pdf/2502.02481">GemmaX2–28–2B</a>, developed by Xiaomi researchers and based on Google’s Gemma-2 architecture.</p><p>To compare them, I ran a series of translation benchmarks using <a href="https://pypi.org/project/sacrebleu/">sacreBLEU</a>, a standardized BLEU (Bilingual Evaluation Understudy) score implementation that ensures reproducibility. I used multilingual datasets such as <a href="https://huggingface.co/datasets/wmt/wmt14">WMT14</a>, <a href="https://huggingface.co/datasets/Helsinki-NLP/europarl">Europarl</a>, and <a href="https://huggingface.co/datasets/Helsinki-NLP/opus-100">Opus-100</a>, testing across key language pairs like English-German, French-English, German-French, and so on. While some of the classic models like Opus-MT performed decently, GemmaX2–28–2B consistently came out on top, in some cases even outperforming DeepL. Most importantly, it did this while remaining small enough (2B parameters) to run inference on a <a href="https://www.nvidia.com/en-us/data-center/tesla-t4/">NVIDIA T4 GPU</a> with reasonable latency.</p><pre>from sacrebleu import corpus_bleu<br><br>hypotheses = [<br>    &quot;This is a translated sentence.&quot;,<br>    &quot;Here is another translation.&quot;<br>]<br><br>references = [<br>    [&quot;This is the reference sentence.&quot;],<br>    [&quot;Here is another reference.&quot;]<br>]<br><br>score = corpus_bleu(hypotheses, references)<br>print(f&quot;sacreBLEU score: {score.score:.2f}&quot;)</pre><pre>❯ python sacrebleu.py<br>sacreBLEU score: 22.96</pre><p>Run this code for each candidate model using parallel reference/test sets (e.g. from Hugging Face datasets like WMT14 or Europarl).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2j3X1ibjom24sWdVn4uP8w.png" /><figcaption>sBLEU scores across several translation models and language pairs.</figcaption></figure><p>I started researching and benchmarking open-source AI models for translation in <strong>November 2024</strong>, at the start of my master thesis project. Over the next few weeks/months, I compared multiple candidates and concluded the research phase by <strong>late March 2025</strong>, just after the release of <strong>GemmaX2–28–2B</strong>.</p><p>The AI landscape moves fast, so if you’re reading this later in 2025 or beyond, newer models may outperform the ones tested here, but the benchmarking approach, latency tuning, and infrastructure lessons will still apply.</p><h4>Manual testing against DeepL</h4><p>Since DeepL doesn’t publish BLEU scores and is essentially a black box, I also included manual testing of the same texts translated via DeepL and GemmaX2. I included HTML content, emojis, incorrect casing, long sentences, and typos to simulate real-life inputs. The results were surprisingly close. In many cases, the LLM-generated translations were indistinguishable from DeepL.</p><h4>Performance matters: latency testing</h4><p>Quality alone isn’t enough. We also needed speed. On initial tests, most open-source models took over 25 seconds to translate a 5,000-character text (maximum size of an article description on Ricardo), clearly unacceptable for production. But with GemmaX2, once we optimized the inference pipeline using <a href="https://docs.vllm.ai/en/latest/">vLLM</a>, sentence tokenization, FP16 quantization, we brought the average response time down to under 3 seconds. That put us within the latency budget we needed for a smooth user experience.</p><p>In the end, GemmaX2–28–2B stood out as the only model that offered both great translation quality and production-grade performance. It’s not the most well-known model out there, but it proved itself where it mattered: translating real, messy, multilingual content quickly and reliably. That made it our clear winner, and the foundation for everything that came next.</p><h3>Optimizations and API design</h3><p>Once the GemmaX2–28–2B model was selected, the real challenge began: making it fast and robust enough for a real-time user-facing service.</p><p>Out of the box, even the best models were too slow. Translating a single 5,000-character text could take over 25 seconds on a T4 GPU. That kind of latency would ruin the user experience. So the next phase of the project was all about optimizing for speed and stability.</p><h4>Reducing latency: from 25s to under 3s</h4><p>The first major win came from <strong>quantizing the model to FP16</strong>, which reduced memory usage and sped up inference significantly. This was a straightforward change, but surprisingly effective.</p><p>By default, most models run with 32-bit floating point (FP32) precision, which offers high numerical accuracy but also consumes a lot of memory and compute. FP16, or 16-bit floating point, cuts that in half, using less GPU memory and allowing more operations to fit in parallel. For inference tasks like translation, FP16 precision is typically more than sufficient, with little to no loss in quality. On GPUs like the NVIDIA T4, using FP16 allows faster throughput, lower latency, and the ability to process larger batches, all without retraining the model.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*kwha4neFqXeR8jFy" /><figcaption>BF16 vs FP32 vs FP16 (<a href="https://www.exxactcorp.com/blog/hpc/what-is-fp64-fp32-fp16">source</a>)</figcaption></figure><p>Next, I integrated the model with <strong>vLLM</strong>, an inference engine designed specifically for large language models. Unlike the default Hugging Face pipelines (like transformers), vLLM is optimized for high-throughput and low-latency tasks. It supports batched execution, streaming responses, and asynchronous inference, which together allow us to fully utilize the GPU.</p><p>But the real breakthrough came from handling long texts more intelligently.</p><p>Instead of feeding entire blocks of HTML or full descriptions to the model, I built a preprocessing pipeline using <a href="https://pypi.org/project/beautifulsoup4/"><strong>BeautifulSoup</strong></a> and <a href="https://www.nltk.org/"><strong>NLTK</strong></a>. The system first extracts all the text nodes from the HTML, then splits the content into individual sentences (<strong>NLTK</strong>) based on the detected source language by <a href="https://pypi.org/project/pycld2/"><strong>pycld2</strong></a> library. Each sentence is translated independently and in parallel using Python’s asyncio.gather, and then the results are reassembled into the original HTML structure. This approach not only preserves formatting, but also significantly speeds up the translation since short sentences are much quicker to process in parallel, than long, unstructured blobs.</p><p>On top of that, I set the model’s <strong>temperature to 0</strong>, which eliminates randomness in generation and results in faster, more deterministic outputs. Since we don’t need creativity in translations, just accuracy, this was a safe and effective tweak.</p><p>With all these optimizations in place, the average response time for long texts dropped from over 25 seconds to under 3, with 95% of requests completing in less than 2.7 seconds. That made the system viable for production use.</p><h4>Wrapping it in a FastAPI microservice</h4><p>With the model optimized, I needed a clean, scalable interface to expose it internally. For that, I built a lightweight microservice using <a href="https://fastapi.tiangolo.com/"><strong>FastAPI</strong></a>.</p><p>The main endpoint is a simple POST route, /translate, which accepts raw HTML content and a target language. The service automatically detects the source language using pycld2, applies the preprocessing pipeline, and returns the translated HTML. In case of unexpected errors, like an unsupported language, a malformed input, or a timeout from the model, the service responds with detailed error messages and appropriate HTTP status codes (400, 500, or 503). These errors are logged and exposed through <a href="https://prometheus.io/">Prometheus</a> for monitoring.</p><pre>from fastapi import FastAPI, HTTPException<br>from pydantic import BaseModel<br>from openai import AsyncOpenAI<br>import re<br>import html<br>import asyncio<br>from bs4 import BeautifulSoup, NavigableString, Tag<br>from nltk.tokenize import sent_tokenize<br>import httpx<br>import requests<br>from prometheus_client import make_asgi_app<br>from app.instrumenting import prometheus_middleware<br>import nltk<br>import pycld2 as cld2<br><br># Download NLTK models for sentence tokenization<br>nltk.download(&quot;punkt&quot;)<br>nltk.download(&quot;punkt_tab&quot;)<br><br>app = FastAPI()<br><br>VLLM_SERVER_URL = &quot;http://localhost:8000/v1&quot;<br>API_KEY = &quot;token-123&quot;<br>vllm_running_status = False<br><br>SUPPORTED_LANGUAGES = {<br>    &quot;en&quot;: &quot;english&quot;,<br>    &quot;fr&quot;: &quot;french&quot;,<br>    &quot;de&quot;: &quot;german&quot;,<br>    &quot;it&quot;: &quot;italian&quot;,<br>    &quot;un&quot;: &quot;unknown&quot;<br>}<br><br>client = AsyncOpenAI(<br>    base_url=VLLM_SERVER_URL,<br>    api_key=API_KEY,<br>)<br><br>class TranslateRequest(BaseModel):<br>    text: str<br>    target_lang: str<br><br>app.middleware(&quot;http&quot;)(prometheus_middleware)<br>app.mount(&quot;/metrics&quot;, make_asgi_app())<br><br>@app.get(&quot;/health&quot;)<br>async def health_check():<br>    global vllm_running_status<br>    if vllm_running_status:<br>        return {&quot;status&quot;: &quot;ok&quot;}<br>    try:<br>        async with httpx.AsyncClient(timeout=3.0) as client:<br>            resp = await client.get(&quot;http://localhost:8000/health&quot;)<br>        if resp.status_code == 200:<br>            vllm_running_status = True<br>            return {&quot;status&quot;: &quot;ok&quot;}<br>        else:<br>            raise HTTPException(status_code=503, detail=&quot;vLLM server is not healthy&quot;)<br>    except requests.RequestException:<br>        raise HTTPException(status_code=503, detail=&quot;vLLM server is down&quot;)<br><br>@app.get(&quot;/ready&quot;)<br>async def ready_check():<br>    try:<br>        async with httpx.AsyncClient(timeout=3.0) as client:<br>            resp = await client.get(&quot;http://localhost:8000/health&quot;)<br>        if resp.status_code == 200:<br>            return {&quot;status&quot;: &quot;ok&quot;}<br>        else:<br>            raise HTTPException(status_code=503, detail=&quot;vLLM server is not ready&quot;)<br>    except requests.RequestException:<br>        raise HTTPException(status_code=503, detail=&quot;vLLM server is down&quot;)<br><br>@app.post(&quot;/translate&quot;)<br>async def translate_text(data: TranslateRequest):<br>    # Clean input<br>    t = safe_utf8(data.text)<br>    t = strip_control_and_bidi(t)<br>    t = unescape_entities(t)<br><br>    # Detect source language<br>    try:<br>        _, _, details = cld2.detect(t)<br>        lang_code = details[0][1].lower()  # use ISO code (e.g., &#39;de&#39;)<br>    except cld2.error:<br>        raise HTTPException(status_code=400, detail=&quot;Could not detect source language&quot;)<br><br>    # Map to NLTK language name<br>    nltk_lang = SUPPORTED_LANGUAGES.get(lang_code)<br>    if nltk_lang is None:<br>        nltk_lang = &quot;unknown&quot;<br><br>    # Validate target language<br>    target_lang = SUPPORTED_LANGUAGES.get(data.target_lang.lower())<br>    if not target_lang:<br>        raise HTTPException(status_code=400, detail=f&quot;Unsupported target language: {target_lang}&quot;)<br><br>    # Perform HTML-preserving translation with sentence splitting<br>    translated = await translate_html(data.text, target_lang, nltk_lang)<br>    return {&quot;translation&quot;: translated}<br><br>def safe_utf8(text: str) -&gt; str:<br>    return text.encode(&quot;utf-8&quot;, errors=&quot;ignore&quot;).decode(&quot;utf-8&quot;, errors=&quot;ignore&quot;)<br><br>def strip_control_and_bidi(text: str) -&gt; str:<br>    return re.sub(r&#39;[\x00-\x1F\u202A-\u202E]&#39;, &#39;&#39;, text)<br><br>def unescape_entities(text: str) -&gt; str:<br>    return html.unescape(text)<br><br># HTML-preserving translation<br>async def translate_text_chunk(text: str, source_lang: str, target_lang: str) -&gt; str:<br>    # Use a different prompt if the source language is unknown<br>    if source_lang == &quot;unknown&quot;:<br>        prompt = f&quot;Translate the following text to {target_lang}:\n{text}\n{target_lang}:&quot;<br>    else:<br>        prompt = f&quot;Translate this from {source_lang} to {target_lang}:\n{source_lang}: {text}\n{target_lang}:&quot;<br>        <br>    response = await client.chat.completions.create(<br>        model=&quot;/model&quot;,<br>        messages=[<br>            {&quot;role&quot;: &quot;user&quot;, &quot;content&quot;: prompt},<br>        ],<br>        temperature=0.0,<br>        top_p=0.5,<br>        stream=True,<br>    )<br>    output = &quot;&quot;<br>    async for chunk in response:<br>        output += chunk.choices[0].delta.content or &quot;&quot;<br>    return output.strip().replace(&quot;\n&quot;, &quot; &quot;)<br><br>async def translate_html(html_text: str, target_lang: str, nltk_lang: str) -&gt; str:<br>    # Use German as the default language for sentence tokenization if the language is unknown<br>    if nltk_lang == &quot;unknown&quot;:<br>        nltk_lang = &quot;german&quot;<br><br>    # Parse already cleaned HTML input<br>    soup = BeautifulSoup(html_text, &quot;html.parser&quot;)<br><br>    # Collect text nodes<br>    nodes = []<br>    def collect(node):<br>        if isinstance(node, NavigableString):<br>            if node.strip():<br>                nodes.append(node)<br>        elif isinstance(node, Tag):<br>            for c in node.contents:<br>                collect(c)<br>    collect(soup)<br><br>    # Create jobs: split each node into sentences based on source language<br>    async def job(node):<br>        text = str(node)<br>        sentences = sent_tokenize(text, language=nltk_lang)<br>        # Translate sentences in parallel<br>        tasks = [asyncio.create_task(translate_text_chunk(s, nltk_lang, target_lang)) for s in sentences]<br>        trans = await asyncio.gather(*tasks)<br>        return node, &quot; &quot;.join(trans)<br><br>    # Run all jobs concurrently<br>    results = await asyncio.gather(*(job(n) for n in nodes))<br><br>    # Patch the tree in place<br>    for node, txt in results:<br>        node.replace_with(NavigableString(txt))<br><br>    # Serialize back to HTML<br>    return str(soup)</pre><p>At this point, the service was fast, reliable, and API-ready. It could translate large, messy texts in real-time. It could handle HTML, emojis, and non-standard unicode.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hAl0V0odeC2gHdewALktQw.png" /><figcaption>Translation request flow: how Web/Mobile clients interact with the translation service, cache, and model inference pipeline. The system preserves HTML, emojis, and other special content.</figcaption></figure><h3>Productionizing the translation-service</h3><p>Having a fast and functional translation API is one thing. But making it reliable under real-world conditions, deployed, monitored, fault-tolerant, and scalable, is a whole different challenge.</p><p>Here’s how I took the service from “it works on my machine” to full production readiness.</p><h4>Docker, Kubernetes, and GPU scaling</h4><p>The first step was containerization. I built a minimal <strong>Docker image</strong> that bundles the FastAPI app, the vLLM server, and the GemmaX2 model (preloaded into /model) to avoid cold-start delays.</p><p>The service is deployed using <strong>Kubernetes</strong> on Google Kubernetes Engine (GKE), with each <strong>pod</strong> running on a dedicated GPU-enabled node. The model is loaded at container startup so the GPU is ready to serve requests as soon as the pod becomes active.</p><pre>FROM nvidia/cuda:12.8.0-base-ubuntu20.04<br><br># Set environment variables to avoid interactive prompts<br>ENV DEBIAN_FRONTEND=noninteractive<br>ENV TORCHDYNAMO_DISABLE=1<br><br># Install Python 3.9 and build tools<br>RUN apt update &amp;&amp; apt install -y \<br>    python3.9 python3.9-distutils python3.9-venv \<br>    curl git build-essential wget cmake ninja-build \<br>    &amp;&amp; rm -rf /var/lib/apt/lists/*<br><br># Set python3.9 as default<br>RUN ln -sf /usr/bin/python3.9 /usr/bin/python &amp;&amp; \<br>    curl -sS https://bootstrap.pypa.io/get-pip.py | python<br><br># Install Python deps<br>COPY requirements.txt requirements.txt<br>RUN pip install --no-cache-dir -r requirements.txt<br><br># Copy app code<br>COPY app /app<br><br># Download the model (or mount it at runtime instead)<br>RUN python -c &quot;from huggingface_hub import snapshot_download; snapshot_download(&#39;ModelSpace/GemmaX2-28-2B-v0.1&#39;, local_dir=&#39;/model&#39;)&quot;<br><br>WORKDIR /<br>EXPOSE 8080 8000<br><br># Run vLLM server<br>CMD [&quot;bash&quot;, &quot;-c&quot;, &quot;vllm serve /model --dtype half --api-key token-123 --disable-log-requests --max-model-len 8192 --max-num-seqs 64 --max-num-batched-tokens 8192  &amp; uvicorn app.main:app --host 0.0.0.0 --port 8080 --workers 1&quot;]</pre><p><em>This Dockerfile builds both the FastAPI server and the vLLM inference engine into a single container, optimized for GPU T4 inference.</em></p><p>For deployment, I used <strong>Google Kubernetes Engine (GKE)</strong> with a dedicated <strong>GPU node pool</strong> running <strong>NVIDIA T4 GPUs</strong>. Each node hosts a single translations-service pod to ensure GPU isolation, and Kubernetes probes (/health, /ready) were added to guarantee that each instance is healthy before receiving traffic.</p><p>The <strong>NVIDIA T4</strong> is particularly well-suited for inference workloads like this. It offers a cost-effective balance between performance and energy efficiency:</p><ul><li>It supports <strong>FP16 </strong>and<strong> INT8 </strong>precision, both ideal for high-throughput inference tasks.</li><li>It has a <strong>thermal design power </strong>(TDP)<strong> of just 70W</strong>, making it far more energy-efficient than larger datacenter GPUs like A100 or H100.</li><li>It’s widely available on cloud platforms (including GCP) at a <strong>low hourly rate</strong>, which helped us stay well under budget.</li></ul><pre>apiVersion: isopod/v1<br>default:<br>  sreCompliant: false<br>  timeoutMinutes: 25<br>  gpu:<br>    enabled: true<br>  appPort: 8080<br>  env:<br>    HTTP_TIMEOUT_S: 5<br>    HTTP_GRACEFUL_SHUTDOWN_WAIT_S: 30<br>    LOGGING_LEVEL: WARN<br>  probes:<br>    startupProbe:<br>      initialDelaySeconds: 500<br>      periodSeconds: 10<br>      failureThreshold: 30<br>      successThreshold: 1<br>    livenessProbe:<br>      httpGet:<br>        path: /health<br>        port: 8080<br>      periodSeconds: 10<br>      failureThreshold: 30<br>      timeoutSeconds: 30<br>    readinessProbe:<br>      httpGet:<br>        path: /ready<br>        port: 8080<br>      periodSeconds: 10<br>      failureThreshold: 30<br>      timeoutSeconds: 30<br>dev:<br>  replicaCount: 1<br>  env:<br>    LOGGING_LEVEL: INFO<br>  istioProxyCPURequest: 50m<br>  resources:<br>    limits:<br>      gpu: 1<br>      memory: 10Gi<br>    requests:<br>      memory: 10Gi<br>metadata:<br>  labels:<br>    app: translations-service<br>    language: python<br>    tiers: backend<br>  language: python<br>  maintainer: purchasing<br>  name: translations-service<br>  role: api<br>prod:<br>  replicaCount: 2<br>  env:<br>    LOGGING_LEVEL: INFO<br>  istioProxyCPURequest: 25m<br>  resources:<br>    limits:<br>      gpu: 1<br>      memory: 10Gi<br>    requests:<br>      memory: 10Gi</pre><p><em>Here’s the full </em><em>isopod.yml file used to deploy the service to GKE with NVIDIA T4 GPU support, Kubernetes probes, and per-environment config.</em></p><p>With this configuration, I was able to deploy a highly performant LLM-backend service at a <strong>fraction of the cost and energy usage</strong> compared to more powerful GPUs, while still meeting latency and scalability targets for production use.</p><pre>name: Release API<br>run-name: Test, Build and Deploy from ${{ github.ref_name }} by @${{ github.actor }}<br><br>on:<br>  push:<br>    paths:<br>      - &#39;**&#39;<br>      - &#39;!**/README.md&#39;<br><br>permissions:<br>  id-token: write<br>  contents: read<br><br>concurrency:<br>  group: ${{ github.workflow }}-${{ github.ref }}<br>  cancel-in-progress: true<br><br>jobs:<br>  quality-gate:<br>    runs-on: blacksmith-2vcpu-ubuntu-2204<br>    steps:<br>      - name: Checkout code<br>        uses: actions/checkout@v4<br>      - name: Setup Python<br>        uses: actions/setup-python@v5<br>        with:<br>          python-version: 3.9<br>      - uses: useblacksmith/setup-python@v6<br>        with:<br>          python-version: 3.9<br>          cache: &#39;pip&#39;<br>      - name: Install dependencies<br>        run: pip install -r requirements.txt<br>      - name: Run tests<br>        run: pytest<br>  <br>  docker:<br>    needs: quality-gate<br>    runs-on: blacksmith-2vcpu-ubuntu-2204<br>    steps:<br>      - name: Checkout code<br>        uses: actions/checkout@v4<br>      - name: GCP Auth<br>        uses: ricardo-ch/auth-action@v1<br>      - name: Build and push Docker image<br>        uses: ricardo-ch/isopod-action@v1<br>        with:<br>          command: build -f ./isopod.yml<br><br>  deploy-dev-branch:<br>    if: ${{ github.ref != &#39;refs/heads/main&#39; }}<br>    needs: docker<br>    runs-on: blacksmith-2vcpu-ubuntu-2204<br>    environment: development<br>    steps:<br>      - name: Checkout code<br>        uses: actions/checkout@v4<br>      - name: GCP Auth<br>        uses: ricardo-ch/auth-action@v1<br>        with:<br>          gke_cluster: dev-banana<br>      - name: Deploy to dev<br>        uses: ricardo-ch/isopod-action@v1<br>        id: deploy<br>        with:<br>          command: deploy --environment dev -f ./isopod.yml<br><br>  deploy-dev-main:<br>    if: ${{ github.ref == &#39;refs/heads/main&#39; }}<br>    needs: docker<br>    runs-on: blacksmith-2vcpu-ubuntu-2204<br>    steps:<br>      - name: Checkout code<br>        uses: actions/checkout@v4<br>      - name: GCP Auth<br>        uses: ricardo-ch/auth-action@v1<br>        with:<br>          gke_cluster: dev-banana<br>      - name: Deploy to dev<br>        uses: ricardo-ch/isopod-action@v1<br>        id: deploy<br>        with:<br>          command: deploy --environment dev -f ./isopod.yml<br>  <br>  deploy-prod-main:<br>    if: ${{ github.ref == &#39;refs/heads/main&#39; }}<br>    needs: deploy-dev-main<br>    runs-on: blacksmith-2vcpu-ubuntu-2204<br>    steps:<br>      - name: Checkout code<br>        uses: actions/checkout@v4<br>      - name: GCP Auth<br>        uses: ricardo-ch/auth-action@v1<br>        with:<br>          gke_cluster: prod-tomato<br>      - name: Deploy to prod<br>        uses: ricardo-ch/isopod-action@v1<br>        id: deploy<br>        with:<br>          command: deploy --environment prod -f ./isopod.yml</pre><p><em>Github Actions workflow that runs tests, builds the image, and deploys the translations-service to GKE.</em></p><p>At this stage, we didn’t use pod-level autoscaling because traditional metrics like CPU or memory weren’t useful, and a single request can fully saturate the GPU. Instead, we relied on node-level autoscaling, with plans to later integrate a custom metric (like vLLM queue length) for smarter horizontal scaling.</p><h4>Caching for speed and cost savings</h4><p>To reduce inference load and improve response times, the translations-api checks a <strong>Redis cache</strong> before calling the model (translations-service). If a translation for the same source text and target language exists, it’s returned instantly. Otherwise, the request goes to the model, and the result is cached with a TTL (Time To Live) of 90 days.</p><p>This caching layer proved essential, not just for speed, but for keeping cloud GPU costs under control. With this in place, only new or changed texts trigger inference, while most traffic is served in milliseconds.</p><h4>Monitoring and observability</h4><p>No production service is complete without observability. I instrumented the FastAPI app with <strong>Prometheus</strong>, exposing custom metrics with the /metrics endpoint. These include the following metrics:</p><ul><li>Request duration histograms, broken down by endpoint and status code</li><li>Client disconnects (e.g. user cancels the request mid-stream)</li></ul><pre>from prometheus_client import Histogram, Counter, REGISTRY<br>from fastapi import Request<br>from fastapi.exceptions import HTTPException<br>import time<br><br># Prometheus metrics<br>REQUEST_DURATION = Histogram(<br>    &#39;http_request_duration_seconds&#39;,<br>    &#39;HTTP request duration in seconds&#39;,<br>    [&#39;component&#39;, &#39;handler&#39;, &#39;method&#39;, &#39;code&#39;, &#39;endpoint_severity&#39;, &#39;uri&#39;],<br>    buckets=[.01, .1, .25, .5, .75, 1, 5, 10]<br>)<br><br>CONNECTION_LOST_TOTAL = Counter(<br>    &#39;http_server_requests_connection_lost_total&#39;,<br>    &#39;Number of client connections that have been closed before the server was able to respond&#39;,<br>    [&#39;source_app&#39;, &#39;handler&#39;, &#39;method&#39;]<br>)<br><br># Register Prometheus metrics if not already registered<br>if &#39;http_request_duration_seconds&#39; not in REGISTRY._names_to_collectors:<br>    REGISTRY.register(REQUEST_DURATION)<br>if &#39;http_server_requests_connection_lost_total&#39; not in REGISTRY._names_to_collectors:<br>    REGISTRY.register(CONNECTION_LOST_TOTAL)<br><br>async def prometheus_middleware(request: Request, call_next):<br>    if request.method == &quot;POST&quot; and request.url.path == &quot;/translate&quot;:<br>        start_time = time.time()<br>        try:<br>            response = await call_next(request)<br>            process_time = time.time() - start_time<br>            REQUEST_DURATION.labels(<br>                component=&quot;translations-service&quot;,<br>                handler=request.url.path,<br>                method=request.method,<br>                code=response.status_code,<br>                endpoint_severity=&quot;low&quot;,<br>                uri=request.url.path<br>            ).observe(process_time)<br>            return response<br>        except Exception as e:<br>            if isinstance(e, HTTPException) and e.status_code == 503:<br>                CONNECTION_LOST_TOTAL.labels(<br>                    source_app=request.client.host,<br>                    handler=request.url.path,<br>                    method=request.method<br>                ).inc()<br>            raise<br>    else:<br>        return await call_next(request)</pre><p>All of these metrics are visualized in <a href="https://grafana.com/"><strong>Grafana</strong></a><strong> dashboards</strong>, which allow the team to monitor the system’s health in real time. For example, we track:</p><ul><li>Success vs error rates</li><li>Latency distributions (p50, p95, p99)</li><li>Request throughput</li><li>Pod restarts and uptime</li></ul><p>We also set up <strong>Slack alerts</strong> via Alertmanager for anomalies, such as spikes in 5xx or 4xx errors. This made the system production-grade not just in performance, but in reliability.</p><h4>Load testing with Grafana k6</h4><p>Before releasing the new service to users, I ran <strong>stress tests using </strong><a href="https://k6.io/"><strong>Grafana k6</strong></a>. The test scenario simulated 20 concurrent users sending 5,000-character texts every second during 40 seconds, a load way higher than our real-world peak (around 3 requests/s).</p><pre>import http from &#39;k6/http&#39;;<br>import { sleep, check } from &#39;k6&#39;;<br><br>export let options = {<br>    stages: [<br>        { duration: &#39;10s&#39;, target: 20 }, // ramp up to 20 users<br>        { duration: &#39;30s&#39;, target: 20 },  // stay at 20 users<br>        { duration: &#39;1s&#39;, target: 0 },  // ramp down to 0 users<br>    ],<br>};<br><br>const longDescription = &quot;Translate this 5000-char text...&quot;;<br><br>export default function () {<br>    const url = &#39;http://localhost:8080/translate&#39;;<br>    const payload = JSON.stringify({<br>        &quot;text&quot;: longDescription,<br>        &quot;target_lang&quot;: &quot;fr&quot;<br>      });<br><br>    const params = {<br>        headers: {<br>            &#39;Content-Type&#39;: &#39;application/json&#39;,<br>        },<br>    };<br><br>    let res = http.post(url, payload, params);<br><br>    check(res, {<br>        &#39;is status 200&#39;: (r) =&gt; r.status === 200,<br>    });<br><br>    sleep(1);<br>}</pre><p>The system handled it well: no pod crashes, no GPU saturation, and all requests completed within a correct latency window (about 30s). These results gave us confidence to move forward with an A/B test in production.</p><h3>Validating the system with A/B testing</h3><p>We had optimized the model, built a fast API, deployed it to production (no trafic redireted to it for now), and set up monitoring. But none of that would matter if our users noticed a drop in translation experience. So before replacing DeepL entirely, we ran a large-scale <strong>A/B test</strong> to validate the solution in the real world.</p><h4>The hypothesis</h4><p>Our goal was simple: prove that users couldn’t tell the difference between DeepL API and our homemade translation system.</p><p>We weren’t just comparing BLEU scores anymore or response times, we were comparing <strong>user</strong> <strong>behavior</strong>. Would users continue using the “Translate” feature on Ricardo? Would it still help them complete purchases or place bids? And would the quality be good enough to support the same outcomes?</p><p>If we saw no drop in usage or conversion, we’d consider the system validated.</p><h4>The Setup</h4><p>The A/B test was conducted on the <strong>Web </strong>platform of Ricardo, using <a href="https://launchdarkly.com/"><strong>LaunchDarkly</strong></a> to split users into two buckets:</p><ul><li>50% of logged in users received translations from <strong>DeepL API</strong></li><li>50% received translations from the new <strong>GemmaX2–28–2B </strong>based service</li></ul><p>The split was transparent, no visual difference, no banners, no user indication of which system was used. We only analyzed sessions where the user actively clicked the “Translate” button on the product detail page. Those were our test population.</p><p>We tracked user behavior using <strong>Google Analytics</strong> and <strong>Tableau</strong>, focusing on session counts and conversion metrics.</p><h4>The results</h4><p>After several weeks in June 2025, the results of the A/B test were remarkably close: the difference in conversion rate between DeepL and our translation system was just <strong>0.22 percentage points</strong>, a difference statistically insignificant.</p><p>This confirmed that users behaved the same regardless of which system provided the translation, and that our in-house model delivered a comparable user experience without any noticeable regression.</p><p>Even better, <strong>no negative feedback</strong> was reported to Customer Care team, and no bugs or regressions were observed on our monitoring dashboards.</p><h4>The decision</h4><p>With the A/B test complete and the data in hand, we met with our Product Manager, Engineering Manager, and Data team. The consensus was clear: the system was ready.</p><p>We rolled out the internal translation service to 100% of traffic on Web, and shortly after, to the iOS and Android apps as well. DeepL was officially removed from our stack, and from our monthly expenses.</p><h3>Results, limitations, and future improvements</h3><p>The rollout of the new translation system wasn’t just a technical win, it delivered <strong>tangible business value</strong>. What started as a cost-reduction initiative turned into a broader transformation of how we handle multilingual content at SMG.</p><h4>Real-world impact</h4><p>The numbers speak for themselves. By replacing DeepL with a self-hosted LLM, we brought our monthly translation infrastructure costs down from over <strong>7,000 CHF </strong>to just under<strong> 1,000 CHF, </strong>a savings of nearly <strong>87%</strong>. These costs include GPU compute on Google Cloud (3 T4 instances) and Redis cache, which remained unchanged.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WnKNRqSB5VDPg8_BH1oBLg.png" /><figcaption><em>GCP billing estimation snapshot showing monthly cost of one NVIDIA T4 node used for inference (n1-standard-4 machine).</em></figcaption></figure><p>In July 2025, our internal translation system processed approximately <strong>2,024,574 translation requests</strong>, with only <strong>8,000 failures</strong>, resulting in a low error rate of about <strong>0.4%</strong>. By contrast, in April 2025, the same workload handled via DeepL API led to <strong>1,766,161 translation requests</strong>, but with a significantly higher error rate of <strong>2.83%</strong> (about <strong>50,000 failures</strong>). While DeepL provides excellent translation quality, these numbers show that our in-house system is not only <strong>more cost-efficient</strong>, but also <strong>more robust and reliable</strong> under our real-world production load.</p><p>Even more important: user experience didn’t suffer. The system maintained a sub-3-second latency for 95% of requests, scored similar on average BLEU evaluations, and showed no statistically significant drop in user conversions during the A/B test. It also translated text reliably, even when faced with emojis, HTML tags, or typo, common traits of user-generated content.</p><h4>A note on sustainability</h4><p>Sustainability was also a consideration in the design of this system.</p><p>By choosing the <strong>NVIDIA T4 GPUs</strong>, we relied on hardware with a <strong>70W TDP</strong>, significantly lower than datacenter-grade alternatives like H100s (350W). T4s offer enough power for optimized LLM inference while remaining energy-efficient, a solid tradeoff between throughput and environmental impact.</p><p>The service was deployed on Google Cloud’s <strong>europe-west1</strong> region, which is certified as a <strong>low-carbon </strong>location. Combined with efficient GPU usage, caching to avoid redundant inference, and no overprovisioning, the system remains relatively light from a resource consumption perspective.</p><p>While it’s not possible to fully quantify the emissions saved compared to a traditional cloud API model like DeepL, every watt counts, and this setup was built with that mindset.</p><h4>Known limitations</h4><p>Of course, the system isn’t perfect, and that’s okay. It’s built to evolve.</p><p>One current limitation is <strong>autoscaling</strong>. While the node pools in GKE are configured to scale based on demand (manually for now), the individual pods don’t yet autoscale intelligently. GPU usage isn’t an ideal metric for scale decisions, and we’re working on exposing better ones, like the number of queued requests in the vLLM server, to drive more responsive scaling.</p><p>Another limitation is <strong>language coverage</strong>. The model currently supports 28 languages, but we’ve only activated four (French, German, Italian and English), the ones used by Ricardo. Expanding to other platforms at SMG, like AutoScout24 or Homegate, will require validating support for additional languages and adapting caching logic per product.</p><p>We also haven’t yet enabled a <strong>fallback mechanism</strong> to the free tier of DeepL. The logic is ready, but since the free plan only allows 500,000 characters per month, we’re holding off until we define a clear emergency strategy. In the meantime, the service is monitored closely to ensure high availability.</p><p>Another important note: while the system matches DeepL in most cases, it’s still <strong>not as robust across all edge cases</strong>. Some complex or noisy texts may take longer to translate or occasionally hit timeouts. In rare cases, the model can hallucinate content or make inconsistent formatting decisions, a known limitation of LLMs that don’t rely on strict alignment training like specialized NMT models. It’s difficult to match a company like DeepL, which has over 1,000 employees, with a homemade project built during a single master thesis. But the goal was never to replicate every feature. It was to build a <strong>practical, cost-efficient, and customizable alternative</strong>,<strong> </strong>and within that scope, the system delivers.</p><p>Lastly, there’s no <strong>user-facing feedback loop</strong> in place. While conversion metrics help us track success at a high level, we’re considering adding a small 👍 / 👎 component below translations to collect qualitative feedback. That would help us detect subtle regressions and improve over time.</p><h4>What’s next?</h4><p>The success of this project opened doors beyond Ricardo. Other teams at SMG are already exploring how they can reuse the same service. Because the system is modular, containerized, and language-agnostic, it can be reused with minimal changes.</p><p>Looking forward, we’re also exploring:</p><ul><li>Nighttime autoscaling or shutdown for dev environments to reduce idle GPU costs</li><li>Committed-use contracts with GCP to lower GPU pricing by 30 to 50%</li><li>Continuous benchmarking against newer open-source models as they emerge</li></ul><p>The foundation is solid. Now it’s about refining, expanding, and continuing to prove that high-performance machine translation doesn’t have to come from a commercial API.</p><h3>Conclusion</h3><p>This project began as a practical challenge: reduce the cost of automatic translations without compromising on quality and response time. But it ended up being much more than that.</p><p>By building and deploying a production-grade translation system based on an open-source LLM, we proved that it’s possible to match the performance of top-tier commercial APIs like DeepL. And we did it with full control over latency, reliability, privacy, and infrastructure cost.</p><p>The final system translates multilingual, user-generated HTML content in under three seconds, handles real production traffic, supports mobile and web clients, and saves thousands of CHF every month. All with no noticeable difference for our users.</p><p>More importantly, this approach is replicable.</p><p>With the right model, tooling (vLLM, FastAPI, Prometheus, etc.), and cloud infrastructure, anyone can build their own internal translation service, whether for a marketplace, any other commercial project, or even a personal project.</p><p>The era of treating LLMs as experimental or “nice to have” is behind us. With the right optimizations and architecture, they’re now production-ready building blocks for real, mission-critical services.</p><p>I hope this article gave you a concrete roadmap to follow if you’re considering replacing a commercial API with something open, customizable, and cost-efficient. And if you’ve already started down that path: good luck. You’ll learn a lot, and you might even surprise yourself with what you can build.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=19b903d00845" width="1" height="1" alt=""><hr><p><a href="https://medium.com/smg-real-estate/build-a-low-cost-low-latency-low-energy-ai-translator-a-deepl-alternative-that-works-19b903d00845">Build a low-cost, low-latency, low-energy AI Translator — A DeepL alternative that works</a> was originally published in <a href="https://medium.com/smg-real-estate">SMG Real Estate Engineering Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Replacing AWS Step Functions with SQS FIFO queues and cutting the cost in half]]></title>
            <link>https://medium.com/smg-real-estate/replacing-aws-step-functions-with-sqs-fifo-queues-and-cutting-the-cost-in-half-9ab97e819b3a?source=rss----2186e5b9bd8f---4</link>
            <guid isPermaLink="false">https://medium.com/p/9ab97e819b3a</guid>
            <category><![CDATA[aws-step-functions]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[cost-optimization]]></category>
            <category><![CDATA[sqs-queue]]></category>
            <dc:creator><![CDATA[Alexei Liulin]]></dc:creator>
            <pubDate>Fri, 14 Mar 2025 14:47:27 GMT</pubDate>
            <atom:updated>2025-03-14T14:47:27.862Z</atom:updated>
            <content:encoded><![CDATA[<p>One of the core services for SMG Real Estate is Search Alerts — a service that allows users to be notified about new properties published on ImmoScout24 and Homegate.</p><p>For example, if a user is looking for a new apartment having N rooms and costing less than X, and not finding any matches now, they can create a search alert with those search criteria. When a new property matching those criteria is published, the user will be notified either via email or a mobile push notification.</p><h3>About the Search Alerts system</h3><p>Users can choose the frequency of notifications — either every 5 minutes or every 4 hours (assuming there are any to be delivered). The vast majority of search alerts are configured with the 5-minute frequency. Just to get an idea of the scale, we have:</p><ul><li>Millions of published properties match saved search alerts every day</li><li>More than a million emails and push notifications sent daily</li></ul><p>Check out our <a href="https://www.elastic.co/blog/homegate-search-experience">“Homegate’s fast and modern search experience helps users find their dream home” blog post</a> for an overview of how Search Alerts work.</p><h3>The Original Implementation</h3><p>In the original implementation we used an AWS Step Function to achieve the 5- minute batching of notifications:</p><ul><li>When a new property matching a search alert arrived, the StartSendNotificationProcess lambda would start a Step Function execution specific to that search alert.</li><li>All it did was waiting for 5 minutes while the matches accumulated in the matches-{searchAlertId} SQS queue, which was programmatically created for that specific search alert</li><li>If more matching listings arrived during the 5-minute waiting period, the StartSendNotificationProcess lambda attempted to start the step function execution with the same name. When it failed with the ExecutionAlreadyExists error, we knew there was already a SF for waiting. That way the deduplication of notification processes was guaranteed.</li><li>After 5 minutes of waiting time, the Step Function execution proceeded with triggering the LoadMatches lambda that received the accumulated matches, generating and sending a notification (either an email or a mobile push)</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*BhayGwVkI3EtpaWj.jpg" /></figure><h3>The Problem?</h3><p>The Step Function’s state machine was defined as follows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/273/0*EGrzYBThf9r1oWuC.jpg" /></figure><p>This workflow corresponds to 4 state transitions. Considering we had over 5M state transitions per day and that Step Functions are billed $0.025 per 1K state transitions, this ended up generating daily costs of hundreds of dollars, which added up to tens of thousands per year.</p><p>So it was only reasonable to try to find a more cost-efficient solution for scheduling.</p><h3>FIFO Queues to the Rescue</h3><p>While brainstorming possible cheaper alternative implementations of scheduling, we considered a number of possibilities including implementing our own scheduling service. But at some point we also realised we could use a combination of properties that AWS SQS FIFO queues offer to achieve exactly the same result at a fraction of the cost. FIFO queues guarantee the message order preservation and exactly-once processing. The second property was particularly interesting for us — FIFO queues have a 5-minute deduplication interval, during which trying to send an identical message will not introduce duplicates into the queue. Because the LoadMatches lambda only needs a search alert ID as input, multiple matches for the same search alerts effectively generated equivalent messages that could be deduplicated. This, in combination with a 5-minute message delay, would allow us to achieve the same 5-minute batching as with Step Functions:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*sAs7R6Ho-0RI7E-w.jpg" /></figure><p>On the pricing side, FIFO queues promised lots of savings: $0.48/Million messages VS $25/Million state transitions.</p><p>It’s worth mentioning that we didn’t consider FIFO queues for the original implementation because they were introduced years after our system was first deployed.</p><h3>Shadowing and Analysis</h3><p>Because it seemed too good to be true, we wanted to make sure the new scheduling approach would be equivalent to the old one and that we wouldn’t lose any notifications along the way.</p><p>To do that we implemented the following setup:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Snt2UodDFIpXwAzX.jpg" /></figure><p>We started to log the scheduling-related info on every execution in the StartSendNotificationProcess lambda. That included a randomly generated scheduleId as well as searchAlertId and processStart timestamp. Every time we got a new match we attempted to start a new Step Function execution, but also sent a message to the newly introduced FIFO queue. The scheduling log info was passed in the payload to both channels Every time the LoadMatches or FakeLoadMatches were invoked, we logged some more scheduling-related info such as scheduleId , searchAlertId and processEnd. FakeLoadMatches did nothing but logging — the idea was to shadow the existing production flow and send the same events through both the old and the new flows and compare the results. We used the scheduling IDs and search alert IDs for correlation and to prove that the new shadow flow with FIFO queue would result in equivalent trigger events for the LoadMatches lambda as the Step Function flow.</p><p>We needed to be able to stitch the logs together and answer the main question — given a search alert, does the FIFO flow produce the same trigger events with the same payload at roughly the same time compared to the existing Step Function flow? To do that, we resorted to AWS Athena. That involved:</p><ul><li>Exporting CloudWatch logs from each lambda into a S3 bucket</li><li>Creating an Athena table from the logs of each lambda</li><li>Using SQL to join, filter and aggregate the data from those tables to get the quantity, time difference and distribution of output trigger events</li></ul><p>One challenge we faced was to build a query for creating an Athena table from the mix of JSON and space-separated info coming from the logger utility we’ve been using. In the end we just went for the simplest solution possible — using the vanilla console.log() from the Node runtime and log the spaces-separated values. This way it was trivial to parse and ingest them into Athena. After running multiple queries we proved that the deduplication and delay in the new flow worked as expected, and the LoadMatches and FakeLoadMatches were triggered with events that would result in equivalent notifications delivered to the users. Now that the new architecture has been validated, we were ready to perform the switch.</p><h3>Gradual Roll-Out</h3><p>We still wanted to be cautious and roll out the new scheduling solution gradually — starting with 1% of the traffic and then increasing it all the way to 100% if the system behaved well. This could be achieved by routing the events in StartSendNotificationProcess to either the Step Function or FIFO flow, and connecting the FIFO queue directly to the LoadMatches lambda, so the latter can be triggered with the same result by both the Step Function and the FIFO flows. For testing purposes, we wanted to guarantee that a given search alert will always go through the same flow — either the FIFO queue or the Step Function. This meant that simply rolling a dice wouldn’t cut it — so we decided to use a RegExp to match against search alert IDs to decide which was to route notifications:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*T1kBqw7gWDmBW9B-.jpg" /></figure><p>We created some identical test search alerts and manipulated their IDs in such a way that they end up using different flows.</p><p>This allowed us to do one last manual test before increasing the rollout percentage — we verified that both search alerts resulted in notification emails containing the same properties. As the system was stable and didn’t produce any errors, we manipulated the RegExp to progressively reach the 100% rollout. After that the Step Function flow was completely removed from the system, and now all the notifications for 5-minute search alerts are going through the one and only FIFO queue flow.</p><h3>Conclusion</h3><p>Using SQS FIFO queues to basically debounce match events for 5 minutes while batching the notifications turned out to be a success. It’s a simple solution that allowed us to achieve the same result at a fraction of cost. Speaking of costs, this improvement alone cut the AWS costs in half:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*WLfHFRztTmF8J5nz.jpg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*jFaQuUgFhfArpGCn.jpg" /></figure><p>Besides the cost cut, the stability of the service also improved — previously, we occasionally got some throttling from the Step Function service when listing and starting new executions in correlation with huge spikes of matches. SQS, on the other hand, has no problem digesting such spikes.</p><p>While this worked great for 5-minute notifications, this approach wouldn’t work for longer batching times, as the 5-minute deduplication interval is hardcoded in SQS FIFO queues. We kept the original Step Function based implementation for those cases, as they really are a tiny amount compared to the 5-minute notifications and the cost impact is negligible.</p><p>However, if one day we decide to get rid of the remaining step functions, we’ll have to find a way of scheduling intervals longer than 5 minutes. One idea could be using scheduled Event Bridge events as recurrent triggers to fetch and send the accumulated matches every X minutes.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9ab97e819b3a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/smg-real-estate/replacing-aws-step-functions-with-sqs-fifo-queues-and-cutting-the-cost-in-half-9ab97e819b3a">Replacing AWS Step Functions with SQS FIFO queues and cutting the cost in half</a> was originally published in <a href="https://medium.com/smg-real-estate">SMG Real Estate Engineering Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AWS API Gateway’s Request Validation: Navigating the Quirks]]></title>
            <link>https://medium.com/smg-real-estate/aws-api-gateways-request-validation-navigating-the-quirks-38285cb0a1c1?source=rss----2186e5b9bd8f---4</link>
            <guid isPermaLink="false">https://medium.com/p/38285cb0a1c1</guid>
            <category><![CDATA[aws-api-gateway]]></category>
            <category><![CDATA[aws-lambda]]></category>
            <category><![CDATA[zod]]></category>
            <category><![CDATA[typescript]]></category>
            <category><![CDATA[middy]]></category>
            <dc:creator><![CDATA[Thomas Klein]]></dc:creator>
            <pubDate>Fri, 09 Aug 2024 12:07:11 GMT</pubDate>
            <atom:updated>2024-08-09T22:03:58.626Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/400/1*9qfyIabN-Hkr7hobcbJlpg.jpeg" /></figure><p>AWS API Gateway’s request validation follows a specific set of rules that developers should be aware of when setting up their APIs. While it offers some validation features, it’s important to understand its scope and limitations. Let’s examine how it works with a common API configuration:</p><pre>parameters:<br>  - name: offerType<br>    in: query<br>    required: true<br>    description: RENT or BUY<br>    schema:<br>      type: string<br>      enum:<br>        - RENT<br>        - BUY</pre><p>With this configuration, API Gateway diligently checks for the presence of the query parameter, returning an HTTP 400 error if it’s missing. However, it falls short when it comes to validating the actual value. Surprisingly, it would accept “KERMIT” without batting an eye, despite the clear enum specification.</p><p>This behavior, while <a href="https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-method-request-validation.html#api-gateway-request-validation-basic-definitions">documented</a>, can be a source of frustration for developers expecting more comprehensive validation.</p><p>Interestingly, API Gateway shows more validation prowess with POST and PUT requests. It validates request payloads using <a href="https://json-schema.org/specification-links#draft-4">JSON Schema Draft 4</a>, demonstrating a somewhat inconsistent approach to input validation across different HTTP methods.</p><p>Even when it does catch an error, API Gateway’s feedback is less than illuminating. Instead of specific error details, you’re met with a generic {“message”: “Invalid request body”}. For more information, you’ll need to dig into the associated log streams, which isn’t ideal for surfacing error details in API responses.</p><figure><img alt="Example output from a failed request validation when a body request validator is setup" src="https://cdn-images-1.medium.com/max/1024/1*WauJADzbYJpCSjLSKtxDcg.png" /><figcaption>Example output in AWS console from a failed request validation when a body request validator is setup.</figcaption></figure><p>API Gateway’s validation is handy, but it leaves a lot to be desired. Developers often need to add extra checks for a truly robust API. While API Gateway supports various integrations and Lambda languages, let’s focus on TypeScript.</p><p>TypeScript’s strong typing and runtime checks can catch issues early, sparing your users from cryptic error messages. Let’s explore how to use this to make your API smarter and your users happier.</p><h3>Option 1: DIY Validation (AKA The “I’ve Got Trust Issues” Approach)</h3><p>The most straightforward approach is implementing validation directly in your Lambda handler:</p><pre>import { APIGatewayProxyEvent, APIGatewayProxyResult } from &#39;aws-lambda&#39;;<br><br>enum OfferType {<br>  RENT = &#39;RENT&#39;,<br>  BUY = &#39;BUY&#39;<br>}<br><br>function validateOfferType(offerType: string | undefined): offerType is OfferType {<br>  return offerType !== undefined &amp;&amp; Object.values(OfferType).includes(offerType as OfferType);<br>}<br><br>export const handler = async (event: APIGatewayProxyEvent): Promise&lt;APIGatewayProxyResult&gt; =&gt; {<br>  try {<br>    const offerType = event.queryStringParameters?.offerType;<br>    if (!validateOfferType(offerType)) {<br>      return {<br>        statusCode: 400,<br>        body: JSON.stringify({ error: &#39;Invalid or missing offerType. Must be either RENT or BUY.&#39; })<br>      };<br>    }<br>    // Your amazing business logic here<br>    return {<br>      statusCode: 200,<br>      body: JSON.stringify({ message: `Valid offerType: ${offerType}` })<br>    };<br>  } catch (error) {<br>    console.error(&#39;Error:&#39;, error);<br>    return {<br>      statusCode: 500,<br>      body: JSON.stringify({ error: &#39;Internal server error&#39; })<br>    };<br>  }<br>};</pre><p>This approach, while functional, has limitations:</p><ol><li>Validation logic must be built for each parameter.</li><li>APIGatewayProxyEvent lacks schema-specific type information.</li><li>TypeScript can’t infer parameter existence, requiring Optional Chaining.</li><li>Unit tests must cover both validation and business logic.</li></ol><p>To address the type information issue, you could extend the APIGatewayProxyEvent interface:</p><pre>…<br>interface ExtendedAPIGatewayProxyEvent extends APIGatewayProxyEvent {<br>  queryStringParameters: {<br>    offerType: OfferType;<br>  } &amp; Record&lt;string, string&gt;;<br>}<br><br>function isExtendedEvent(event: APIGatewayProxyEvent): event is ExtendedAPIGatewayProxyEvent {<br>  return (<br>    event.queryStringParameters !== null &amp;&amp;<br>    &#39;offerType&#39; in event.queryStringParameters &amp;&amp;<br>    Object.values(OfferType).includes(event.queryStringParameters.offerType as OfferType)<br>  );<br>}<br>export const handler = async (event: APIGatewayProxyEvent): Promise&lt;APIGatewayProxyResult&gt; =&gt; {<br>  if (!isExtendedEvent(event)) {<br>    return {<br>      statusCode: 400,<br>      body: JSON.stringify({ error: &#39;Invalid or missing offerType. Must be either RENT or BUY.&#39; })<br>    };<br>  }<br>  <br>  const { offerType } = event.queryStringParameters;<br>  // Proceed with business logic<br>};<br>…</pre><p>While this improves type safety, it still requires extensive validation code within the handler. Ideally, we should explore methods that allow our handler to work with pre-validated parameters from the start.</p><h3>Option 2: Middleware Magic (AKA “Let Someone Else Deal With It”)</h3><p>To improve our validation strategy, we can move validation logic outside the handler using middleware. This example uses the popular Middy framework (<a href="https://middy.js.org/">https://middy.js.org/</a>) for AWS Lambda:</p><pre>import middy from &#39;@middy/core&#39;<br>import httpErrorHandler from &#39;@middy/http-error-handler&#39;<br>import { APIGatewayProxyEvent, APIGatewayProxyResult } from &#39;aws-lambda&#39;<br><br>enum OfferType {<br>  RENT = &#39;RENT&#39;,<br>  BUY = &#39;BUY&#39;<br>}<br><br>interface ValidatedEvent extends Omit&lt;APIGatewayProxyEvent, &#39;queryStringParameters&#39;&gt; {<br>  queryStringParameters: {<br>    offerType: OfferType;<br>  }<br>}<br><br>const validateInput = (): middy.MiddlewareObj&lt;APIGatewayProxyEvent, APIGatewayProxyResult&gt; =&gt; ({<br>  before: async (request) =&gt; {<br>    const { event } = request<br>    const offerType = event.queryStringParameters?.offerType<br><br>    if (!offerType || !Object.values(OfferType).includes(offerType as OfferType)) {<br>      throw new Error(&#39;Invalid or missing offerType. Must be either RENT or BUY.&#39;)<br>    }<br><br>    (request as any).event = event as ValidatedEvent<br>  }<br>})<br><br>const baseHandler = async (event: ValidatedEvent): Promise&lt;APIGatewayProxyResult&gt; =&gt; {<br>  const { offerType } = event.queryStringParameters<br>  return {<br>    statusCode: 200,<br>    body: JSON.stringify({ message: `Valid offerType: ${offerType}` })<br>  }<br>}<br><br>export const handler = middy(baseHandler)<br>  .use(validateInput())<br>  .use(httpErrorHandler())</pre><p>This approach offers several advantages:</p><ol><li>Validation occurs before the main handler logic, ensuring type safety.</li><li>The base handler can focus solely on business logic.</li><li>Error handling is normalized using Middy’s httpErrorHandler.</li><li>The ValidatedEvent interface provides strong typing for validated parameters.</li></ol><p>Additionally, this middleware approach can be extended to validate response payloads, further enhancing API reliability.</p><h4>Refining the validation setup</h4><p>Writing custom validation logic for different value types remains a tedious task. You start to think, “I could easily abstract a few things and create a sleek, general-purpose validation library. How long could it possibly take? A few days, maybe?” The possibilities excite you. Everyone in your company could use it! Why not make it your first open-source project? Eternal fame seems just within reach!</p><p>But then, the voice of experience — your inner senior developer — whispers a sobering question: “Have you checked if there are already projects like this out there?”</p><p>And indeed, the npm ecosystem is rich with battle-tested validation libraries. Let’s explore some popular options:</p><ul><li><strong>Zod</strong> (<a href="https://zod.dev/">https://zod.dev/</a>) — via <a href="https://github.com/lechodiman/middy-zod-validator">middy-zod-validator</a></li><li><strong>@middy/validator</strong> with <strong>@middy/validator/transpile</strong> (<a href="https://middy.js.org/docs/middlewares/validator/">https://middy.js.org/docs/middlewares/validator/</a>)</li><li><strong>AJV</strong> (Another JSON Schema Validator) (<a href="https://ajv.js.org/">https://ajv.js.org/</a>) — via <a href="https://github.com/willfarrell/middy-rds">middy-ajv</a></li><li><strong>Joi</strong> (<a href="https://joi.dev/">https://joi.dev/</a>) — via <a href="https://www.npmjs.com/package/middy-sparks-joi">middy-sparks-joi</a></li></ul><p>Each of these libraries has its strengths. While @middy/validator and AJV validate input against JSON schemas, Zod and Joi offer their own APIs for schema setup and complex evaluations.</p><h4>Zod: Streamlining TypeScript Validation</h4><p>For TypeScript enthusiasts, Zod stands out due to its robust type inference capabilities. Let’s refactor our earlier example using Zod:</p><pre>import middy from &#39;@middy/core&#39;<br>import httpErrorHandler from &#39;@middy/http-error-handler&#39;<br>import { APIGatewayProxyEvent, APIGatewayProxyResult } from &#39;aws-lambda&#39;<br>import { z } from &#39;zod&#39;<br>import { zodValidator } from &#39;middy-zod-validator&#39;<br><br>enum OfferType {<br>  RENT = &#39;RENT&#39;,<br>  BUY = &#39;BUY&#39;<br>}<br>const queryStringSchema = z.object({<br>  offerType: z.nativeEnum(OfferType)<br>})<br>type ValidatedEvent = Omit&lt;APIGatewayProxyEvent, &#39;queryStringParameters&#39;&gt; &amp; {<br>  queryStringParameters: z.infer&lt;typeof queryStringSchema&gt;<br>}<br>const baseHandler = async (event: ValidatedEvent): Promise&lt;APIGatewayProxyResult&gt; =&gt; {<br>  const { offerType } = event.queryStringParameters<br>  return {<br>    statusCode: 200,<br>    body: JSON.stringify({ message: `Valid offerType: ${offerType}` })<br>  }<br>}<br>export const handler = middy(baseHandler)<br>  .use(zodValidator({<br>    queryStringParameters: queryStringSchema<br>  }))<br>  .use(httpErrorHandler())</pre><p>This setup provides comprehensive error messages, such as:</p><pre>{<br>  &quot;statusCode&quot;: 400,<br>  &quot;body&quot;: JSON.stringify({<br>    &quot;error&quot;: &quot;Bad Request&quot;,<br>    &quot;message&quot;: &quot;Validation error&quot;,<br>    &quot;details&quot;: [<br>      {<br>        &quot;code&quot;: &quot;invalid_enum_value&quot;,<br>        &quot;options&quot;: [&quot;RENT&quot;, &quot;BUY&quot;],<br>        &quot;path&quot;: [&quot;queryStringParameters&quot;, &quot;offerType&quot;],<br>        &quot;message&quot;: &quot;Invalid enum value. Expected &#39;RENT&#39; | &#39;BUY&#39;, received &#39;KERMIT&#39;&quot;<br>      }<br>    ]<br>  })<br>}</pre><h4>Mission Accomplished</h4><p>With this approach, we’ve achieved our key objectives:</p><ol><li>Separation of validation and business logic.</li><li>Leveraging TypeScript’s type system for handler safety.</li><li>Detailed, helpful error messages for API consumers.</li><li>A reusable, extensible solution thanks to Zod’s schema system.</li></ol><p>Bonus: For those using Powertools for AWS Lambda (<a href="https://docs.powertools.aws.dev/lambda/typescript/latest/">https://docs.powertools.aws.dev/lambda/typescript/latest/</a>), the recent v2.6.0 release includes a parser feature supporting Zod v3+, offering a similar solution to what we’ve discussed.</p><h3>Navigating the Horizon: Where to Next?</h3><p>There’s always room for improvement and further considerations:</p><ol><li>Performance: While the middleware approach is clean and modular, it does add some overhead to each request. For most applications, this overhead is negligible, but for high-throughput APIs, you might want to benchmark and optimize.</li><li>Testing: With this setup, it becomes easier to unit test your business logic separately from your validation logic. Consider writing tests for both your Zod schemas and your handler functions</li><li>Documentation: Now that you have well-defined schemas for your inputs, you could potentially use these to generate API documentation automatically. Examples include <a href="https://www.npmjs.com/package/json-schema-to-zod">json-schema-to-zod</a> or <a href="https://www.npmjs.com/package/zod-to-json-schema">zod-to-json-schema</a>.</li></ol><p>Remember, the goal is to balance security, performance, and developer experience. This solution provides a solid foundation, but always be prepared to adapt and evolve your approach as your API grows and your requirements change.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/667/1*d1EmA6MaA37jPc_DeUta8A.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=38285cb0a1c1" width="1" height="1" alt=""><hr><p><a href="https://medium.com/smg-real-estate/aws-api-gateways-request-validation-navigating-the-quirks-38285cb0a1c1">AWS API Gateway’s Request Validation: Navigating the Quirks</a> was originally published in <a href="https://medium.com/smg-real-estate">SMG Real Estate Engineering Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Update State Management with Pinia]]></title>
            <link>https://medium.com/smg-real-estate/update-state-management-with-pinia-0d856a417535?source=rss----2186e5b9bd8f---4</link>
            <guid isPermaLink="false">https://medium.com/p/0d856a417535</guid>
            <category><![CDATA[state-management]]></category>
            <category><![CDATA[vuejs]]></category>
            <category><![CDATA[pinia]]></category>
            <dc:creator><![CDATA[Dusko Peric]]></dc:creator>
            <pubDate>Thu, 21 Mar 2024 14:43:21 GMT</pubDate>
            <atom:updated>2024-03-21T16:34:33.073Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jb9SKgufCfNFHIWY0yGdFg.jpeg" /></figure><p>In the development and maintenance of web applications, one of the crucial factors is the proper choice of state management. When it comes to Vue.js, we have two players in the game, where in my opinion, we have a clear winner. In this article, I will go through the functionalities brought by Pinia, which represent a step forward in this field.</p><h3><strong>Intuitive API</strong></h3><p>One of the problems in working with state is that it can very easily become confusing and complex. With its component-like structure, Pinia allows for easier understanding of how the store functions and interacts with your application. Let’s look at an example to compare a Vue component with a Pinia store.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/838/1*AGTUirnPdHXxsU2r-1KJmQ.jpeg" /><figcaption>Syntax similarity between Vue component and Pinia store</figcaption></figure><p>This familiar syntax makes it a user-friendly and efficient solution for managing state.</p><h3><strong>Excluding Mutations</strong></h3><p>One of the key differences between Pinia and Vuex lies in the absence of mutations. In Vuex, the distinction between actions and mutations can become less clear-cut, particularly for complex stores. This redundancy can lead to repetitive code, adding unnecessary complexity to state management.</p><p>Pinia takes a more direct approach. You can simply modify the state directly within actions. Let’s see a code example to illustrate this concept:</p><p><em>Vuex Store</em></p><pre>import { createStore } from &#39;vuex&#39;;<br><br>export default createStore({<br>  state: {<br>    count: 0,<br>  },<br>  mutations: {<br>    increment(state) {<br>      state.count++; // Update state through mutation<br>    },<br>  },<br>  actions: {<br>    increment({ commit }) {<br>      commit(&#39;increment&#39;); // Dispatch mutation to update state<br>    },<br>  },<br>});</pre><p><em>Pinia Store</em></p><pre>import { defineStore } from &#39;pinia&#39;;<br><br>export const useCounterStore = defineStore(&#39;counter&#39;, {<br>  state: () =&gt; ({<br>    count: 0,<br>  }),<br>  actions: {<br>    increment() {<br>      this.count++; // Directly update state within action<br>    },<br>  },<br>});</pre><p>In essence, Pinia removes an unnecessary layer of complexity by allowing direct state updates within actions.</p><h3><strong>Modular stores design</strong></h3><p>Pinia allow you to create multiple stores for different parts of your application, each of them should be in their own file, which will help keep code organized. Also is posibile sharing state, actions and getters via nested stores.</p><p>This modularity allows for independent development and testing.</p><h3><strong>Flexibility</strong></h3><p>Pinia offers two main approaches to define stores: Option Stores and Setup Stores.</p><p>While Option Stores resemble Vue’s options API, Setup Stores leverage the power of the Composition API for more flexibility.</p><p><em>Option Store</em></p><pre>import { defineStore } from &#39;pinia&#39;;<br><br>export const useShoppingCart = defineStore(&#39;shoppingCart&#39;, {<br>  state: () =&gt; ({<br>    items: [],<br>  }),<br>  actions: {<br>    addItem(item) {<br>      this.items.push(item);<br>    },<br>  },<br>  getters: {<br>    totalItems() {<br>      return this.items.length;<br>    },<br>  },<br>});</pre><p><em>Setup Store:</em></p><pre>import { defineStore } from &#39;pinia&#39;;<br><br>export const useShoppingCart = defineStore(&#39;shoppingCart&#39;, () =&gt; {<br>  const items = ref([]);<br><br>  const addItem = (item) =&gt; {<br>    items.value.push(item);<br>  };<br><br>  const totalItems = computed(() =&gt; items.value.length);<br><br>  return { items, addItem, totalItems };<br>});</pre><p>Choose the type of store that feels right to you and your use cases.</p><h3>Summary</h3><p>What’s also worth mentioning is that Pinia has built-in TypeScript support and also supports Vue Devtools, which significantly simplifies debugging. Its lightweight nature (around 1.5kb) ensures minimal impact on your application’s performance.</p><p>Pinia offers a refreshing alternative<strong> </strong>for Vue state management, with its focus on simplicity, flexibility, and developer experience. Dive into Pinia and discover its potential to elevate your next project!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0d856a417535" width="1" height="1" alt=""><hr><p><a href="https://medium.com/smg-real-estate/update-state-management-with-pinia-0d856a417535">Update State Management with Pinia</a> was originally published in <a href="https://medium.com/smg-real-estate">SMG Real Estate Engineering Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Automating Multi-Account Permission Management with AWS IAM Identity Center (previously AWS SSO)]]></title>
            <link>https://medium.com/smg-real-estate/automating-multi-account-permission-management-with-aws-iam-identity-center-previously-aws-sso-b1d85281963?source=rss----2186e5b9bd8f---4</link>
            <guid isPermaLink="false">https://medium.com/p/b1d85281963</guid>
            <category><![CDATA[aws-organizations]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[cloudformation]]></category>
            <category><![CDATA[aws-iam]]></category>
            <category><![CDATA[aws-eventbridge]]></category>
            <dc:creator><![CDATA[Niko Jernejsek]]></dc:creator>
            <pubDate>Thu, 22 Dec 2022 10:32:09 GMT</pubDate>
            <atom:updated>2022-12-22T10:43:12.159Z</atom:updated>
            <content:encoded><![CDATA[<h3>Background</h3><p>Homegate was founded in 2001 and is now the leading property marketplace in Switzerland. Few years ago we decided to re-architect our monolithic real estate platform towards micro services on top of AWS cloud. At the moment, Homegate workloads are running almost exclusively on server-less compute model with teams organised around micro services. We are utilising so called multi-account strategy where we structure our AWS accounts around individual micro services further breaking them down into different accounts according to software lifecycle, and organising them into the AWS Organizational Units.</p><p>We’ve structured our AWS Organizations Units &amp; AWS accounts according to the recommendations provided by AWS (<a href="https://docs.aws.amazon.com/whitepapers/latest/organizing-your-aws-environment/basic-organization.html#basic-organization-with-cicd-as-a-separate-function">https://docs.aws.amazon.com/whitepapers/latest/organizing-your-aws-environment/basic-organization.html#basic-organization-with-cicd-as-a-separate-function</a>). Apart from having product workload accounts we also have accounts which are necessary to operate security, governance and platform related workloads.</p><p>Access to accounts is governed by a permission matrix which is based on least privilege principles. The permission matrix classifies who (internally defined personas, e.g.: engineers/managers/security officers, etc) has what type of access (e.g. read-only or admin), and to what category of AWS accounts (e.g. production workloads, shared infra accounts, etc). Group of accounts in our Permission Matrix are at the same time AWS Organisational Units that we implement. As we have to manage permissions for hundreds of engineers to about hundred of AWS accounts accordingly, this way of grouping makes it easier for us to manage it at scale.</p><h3>IAM Identity Center Primer</h3><h4>IAM Identity Center Intro</h4><p>Note that I will be using AWS IAM Identity Center and AWS SSO interchangeably as even some services (such as CloudFormation) are still using the previous name AWS SSO.</p><p>AWS IAM Identity Center is a single sign-on (SSO) service that allows AWS users to centrally manage access and permissions to AWS accounts and applications. Among others, it can connect to external identity providers (IdPs) through the Security Assertion Markup Language (SAML) 2.0 standard. Another important feature of IAM Identity Center is that it integrates with AWS Organization.</p><p>Why did we decided to use IAM Identity Center?</p><ul><li>Centrally manage access and permissions</li><li>External identity support, SAML 2.0 (Okta)</li><li>Integration with AWS Organizations</li><li>Supports SCIM (System for Cross-domain Identity Management)</li><li>No additional cost involved</li></ul><h4>IAM Identity Center (AWS SSO) primitives</h4><p><strong>Users &amp; Groups</strong></p><p>Users are individual identities that can have assigned access to AWS accounts, and groups are a logical combination of users. In our setup of IAM Identity Center, users &amp; groups are provisioned by external identity provider Okta.</p><p>Alternatively, users &amp; groups can be provisioned directly in IAM Identity Center, Active Directory Connector etc.</p><p><strong>Permission Sets</strong></p><p>A permission set is a collection of one or more IAM policies. Explanation on what happens in the background, from AWS documentation:</p><blockquote>When you use AWS SSO to assign a user or group access to one or more AWS accounts, you specify a permission set to define the access that those users or groups will have in the selected AWS accounts. At that time, AWS SSO creates corresponding AWS SSO-controlled IAM roles in each account, and attaches the policies specified in the permission set to those roles. AWS SSO manages the role, and allows the authorized users you’ve defined to assume the role, via the AWS SSO User Portal or AWS CLI. As you modify the permission set, AWS SSO ensures that the corresponding IAM policies and roles are updated accordingly.</blockquote><p><strong>Assignments</strong></p><p>You can think of assignments as a glue which connect users to AWS account with certain permissions. When creating a new assignment, AWS account access is assigned to a principal with specified Permission set, where principal could be either User or a Group.</p><p>ℹ️ Note that IAM Identity Center currently only supports connecting principals directly to AWS Accounts. Organizational Units (OU’s) cannot be used as target for assignment.</p><h3>Challenges</h3><h4>Missing OU’s Support in Assignment Target Types</h4><p>Our Permission matrix is aligned with the OU structure (Organisational Unit) that defines our AWS Organization. Meaning we mostly assign certain set of IAM permissions (Permission Sets) to group of users to group of AWS accounts that are part of specific OU.</p><p>As mentioned earlier, IAM Identity Center doesn’t support AWS Organizational Units (OU’s) as the target type of assignments. That means we now have to make assignments for each individual account, instead of utilising AWS Organizational OU’s as a logical group of AWS accounts which could share the same permissions.</p><p><strong>AWS SSO Assignment </strong><a href="https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-sso-assignment.html"><strong>CloudFormation example</strong></a><strong>:</strong></p><pre>Assignment:<br>    Type: AWS::SSO::Assignment<br>    Properties:<br>      InstanceArn: &#39;arn:aws:sso:::instance/ssoins-instanceId&#39;<br>      PermissionSetArn: !GetAtt PermissionSet.PermissionSetArn<br>      TargetId: &#39;accountId&#39;<br>      TargetType: &#39;AWS_ACCOUNT&#39;<br>      PrincipalType: &#39;USER&#39;<br>      PrincipalId: &#39;user_id&#39;</pre><p>As you can see the supported Target Types only allowed value is AWS Account (at time of writing):</p><pre>TargetType<br> Required: Yes<br> Type: String<br> Allowed values: AWS_ACCOUNT<br> Update requires: Replacement</pre><h4>Solution to Missing OU Target Type Support</h4><p>We decided that extending CloudFormation with a custom logic using <a href="https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-macros.html">AWS CloudFormation macros</a> would be the way forward.</p><p>Luckily for us, solution for this was already present. Ben Kehoe’s <a href="https://github.com/benkehoe/aws-sso-util/tree/master/macro">aws-sso-util</a> implementation of CloudFormation macro addresses our problem. If you want to learn more about CloudFormation macro’s for custom processing, you can check out AWS Documentation (<a href="https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-macros.html">link</a>).</p><p><strong>Over-simplified example about how aws-sso-util extends the AWS SSO CloudFormation functionality:</strong></p><ul><li>Let say we have AWS OU with ID ou-1234-xxxx which currently contains three AWS accounts (111111111111, 222222222222, 333333333333)</li><li>We want to assign same group of users to all three accounts that are members of the OU with same permission sets</li></ul><p>By utilising aws-sso-util’s CloudFormation resource SSOUtil::SSO::AssignmentGroup we can achieve this:</p><pre>AWSTemplateFormatVersion: 2010-09-09<br>Transform: AWS-SSO-Util-2020-11-08<br>Description: &quot;SSO Assignments Group Example&quot;<br>Parameters:<br>  SSOInstanceArn:<br>    Type: String<br>    Description: Instance Arn<br>  PermissionSetExampleArn:<br>    Type: String<br>    Description: Permission Set<br>  SSOGroup:<br>    Type: String<br>    Description: SSO Group<br>Resources:<br> AssignmentGroupExample:<br>   Type: SSOUtil::SSO::AssignmentGroup<br>   Properties:<br>     Name: AssignmentGroupExample<br>     InstanceArn: {&quot;Ref&quot;: &quot;SSOInstanceArn&quot;}<br>     Principal:<br>     - Type: GROUP<br>       Id:<br>       - {&quot;Ref&quot;: &quot;SSOGroupExample&quot;}<br>     PermissionSet: {&quot;Ref&quot;: &quot;PermissionSetExampleArn&quot;}<br>     Target:<br>     - Type: AWS_OU<br>       Id: ou-1234-xxxx</pre><p>Output of macro processing would be CloudFormation template similar to the example below. As you can see the processed template contains three different AWS SSO assignments with “standard” AWS SSO CloudFormation construct AWS::SSO::Assignment with some extra metadata which I omitted for the purpose of readability.</p><pre>AWSTemplateFormatVersion: 2010-09-09<br>Description: &quot;SSO Assignment Group Example&quot;<br>Parameters:<br>  SSOInstanceArn:<br>    Type: String<br>    Description: Instance Arn<br>  PermissionSetExampleArn:<br>    Type: String<br>    Description: Permission Set<br>  SSOGroup:<br>    Type: String<br>    Description: SSO Group<br>AssignmentGroupAssignment1111:<br>  Type: AWS::SSO::Assignment<br>  Properties:<br>    InstanceArn:<br>      Ref: SSOInstanceArn<br>    PrincipalType: GROUP<br>    PrincipalId:<br>      Ref: SSOGroup<br>    PermissionSetArn:<br>      Ref: PermissionSetExampleArn<br>    TargetType: AWS_ACCOUNT<br>    TargetId: &#39;111111111111&#39;<br>AssignmentGroupAssignment2222:<br>  Type: AWS::SSO::Assignment<br>  Properties:<br>    InstanceArn:<br>      Ref: SSOInstanceArn<br>    PrincipalType: GROUP<br>    PrincipalId:<br>      Ref: SSOGroup<br>    PermissionSetArn:<br>      Ref: PermissionSetExampleArn<br>    TargetType: AWS_ACCOUNT<br>    TargetId: &#39;222222222222&#39;<br>AssignmentGroupAssignment3333:<br>  Type: AWS::SSO::Assignment<br>  Properties:<br>    InstanceArn:<br>      Ref: SSOInstanceArn<br>    PrincipalType: GROUP<br>    PrincipalId:<br>      Ref: SSOGroup<br>    PermissionSetArn:<br>      Ref: PermissionSetExampleArn<br>    TargetType: AWS_ACCOUNT<br>    TargetId: &#39;333333333333&#39;</pre><h4>Challenge of Event-driven Approach for Permission Management Automation</h4><p>Solving the challenge of missing direct OU assignment in CloudFormation was the first step in achieving end to end automation. Next we had to find a way to make sure that newly created accounts would also be covered by the CloudFormation SSO assignments stack.</p><p>Now we had to decide about the architecture of the solution which would connect all the dots together and automate the process. We thought of two different architecture approaches that we could implement, either using time driven events (cron based approach), meaning triggering of process periodically, or using more modern approach of event-driven architecture where event we listen for is a change of state in our environment. We decided to go forward with the latter, as speed was one of the deciding factors when choosing between the approaches.</p><h4>Solution to Event-driven Approach — using MoveAccount Event and AWS EventBridge</h4><p>The event that constitutes as a trigger for our process was identified to be MoveAccount event with source being AWS Organizations. MoveAccount occurs every time AWS account is moved between or placed in an OU. This event is captured by AWS CloudTrail.</p><p>We are listening for this event with an AWS EventBridge in the us-east-1 region, this is where events are generated for AWS Organization.</p><p>As most of our workloads are placed in eu-central-1 region, including AWS IAM Identity Center, but events are generated in us-east-1 we are forwarding event from EventBridge to the EventBridge in eu-central-1 region.</p><p>Due to lack of support for cross-regional forwarding of the events in the past, we previously introduced additional SNS topic in between to forward those events from us-east-1 to eu-central-1. Since then AWS extended support for most of the regions (<a href="https://aws.amazon.com/about-aws/whats-new/2021/11/amazon-eventbridge-cross-region-expands/">Amazon EventBridge cross-Region support now expands to more Regions</a>).</p><p>Note, you will need to have CloudTrail’s trail logging events in us-east-1 region in order to capture <em>MoveAccount</em>, regardless if they appear in CloudTrail’s Event history.</p><p><strong>Architecture Diagram</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/901/0*lMINWhbh9oALqWaC" /><figcaption>Architecture Diagram — Automating Multi-Account Permission Management with AWS IAM Identity Center</figcaption></figure><p><strong>Process Flow:</strong></p><ol><li>AWS EventBridge Rule captures AWS Organizations MoveAccount event from AWS CloudTrail (us-east-1).</li><li>AWS EventBridge Rule in us-east-1 forwards event to EventBridge in eu-central-1 region.</li><li>EventBridge rule in eu-central-1 triggers Deployment Author Lambda</li><li>Lambda processes the event from EventBridge, invokes the CloudFormation deployment</li><li>Macro’s lambda gets triggered and generates processed template with AWS SSO assignments</li><li>AWS SSO Assignments are provisioned</li></ol><p><strong>AWS EventBridge event pattern for capturing of MoveEvent originating from AWS Organization:</strong></p><pre>{<br>  &quot;detail-type&quot;: [&quot;AWS API Call via CloudTrail&quot;],<br>  &quot;source&quot;: [&quot;aws.organizations&quot;],<br>  &quot;detail&quot;: {<br>    &quot;eventSource&quot;: [&quot;organizations.amazonaws.com&quot;],<br>    &quot;eventName&quot;: [&quot;MoveAccount&quot;]<br>  }<br>}</pre><h3>Conclusion</h3><p>CloudFormation and AWS services in general won’t always have full out of the box support for features that we require. Luckily AWS provides us with building blocks, such as Macros where we can extend the functionality to support custom processing. And make sure to check for readily available open source option, as most likely others have also faced similar challenges in the past.</p><h4>More about the topics covered in this post:</h4><ul><li><a href="https://docs.aws.amazon.com/singlesignon/latest/userguide/understanding-key-concepts.html">https://docs.aws.amazon.com/singlesignon/latest/userguide/understanding-key-concepts.html</a></li><li><a href="https://docs.aws.amazon.com/singlesignon/latest/userguide/what-is.html">https://docs.aws.amazon.com/singlesignon/latest/userguide/what-is.html</a></li><li><a href="https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-cross-account.html">https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-cross-account.html</a></li><li><a href="https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-event-patterns-content-based-filtering.html">https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-event-patterns-content-based-filtering.html</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b1d85281963" width="1" height="1" alt=""><hr><p><a href="https://medium.com/smg-real-estate/automating-multi-account-permission-management-with-aws-iam-identity-center-previously-aws-sso-b1d85281963">Automating Multi-Account Permission Management with AWS IAM Identity Center (previously AWS SSO)</a> was originally published in <a href="https://medium.com/smg-real-estate">SMG Real Estate Engineering Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Micro Service Events (AWS EventBridge vs. SNS)]]></title>
            <link>https://medium.com/smg-real-estate/micro-service-events-aws-eventbridge-vs-sns-2d86b5919c22?source=rss----2186e5b9bd8f---4</link>
            <guid isPermaLink="false">https://medium.com/p/2d86b5919c22</guid>
            <category><![CDATA[microservice-architecture]]></category>
            <category><![CDATA[aws-sns]]></category>
            <category><![CDATA[eventbus]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[eventbridge]]></category>
            <dc:creator><![CDATA[Kristian Müller]]></dc:creator>
            <pubDate>Fri, 16 Sep 2022 12:06:51 GMT</pubDate>
            <atom:updated>2022-09-16T12:06:50.950Z</atom:updated>
            <content:encoded><![CDATA[<p>During the last few years, we’ve been busy rebuilding the Homegate platform in an event driven, micro service architecture based on AWS. One of the most important buildings blocks, or shall we rather say the mortar that connects the building blocks, is the way our services send events to each other.</p><p>While Amazon offers its <a href="https://en.wikipedia.org/wiki/Amazon_Simple_Notification_Service">notification service SNS</a> since <a href="https://aws.amazon.com/de/blogs/aws/introducing-the-amazon-simple-notification-service/">more than 12 years</a>; in 2019 it started a newcomer in the messaging bus field by introducing <a href="https://aws.amazon.com/de/eventbridge/">EventBridge</a>. This was very interesting to us as we’re using SNS a lot.</p><figure><img alt="Bus on a Bridge" src="https://cdn-images-1.medium.com/max/1024/1*yJp_oA6xOwQ5u-qdh0dDgQ.jpeg" /><figcaption>Bus on a Bridge (Photo by “<a href="https://commons.wikimedia.org/wiki/User:Blue_Elf">Blue Elf</a>” on<a href="https://unsplash.com/s/photos/tools?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText"> </a><a href="https://commons.wikimedia.org/wiki/File:Bus_on_Raftsund_Bridge.jpg">Wikimedia Commons</a>)</figcaption></figure><p>As EventBridge was introduced and gained traction we asked ourselves what advantaged we would gain by switching our SNS based service glue to EventBridge.</p><p><strong>Maybe this turned out to be the wrong question for us. </strong>Let’s look at our system to see why.</p><h3>Micro Services Connected by SNS</h3><p>Homegate is connecting dozens of services via SNS right now. Those services range from single purpose Lambda functions written in TypeScript to database heavy services using Aurora or DynamoDB and <a href="https://aws.amazon.com/de/step-functions">Step Functions</a> sending events to other services. Some of the services are also interfacing external APIs such as Salesforce or SendGrid. Here our Lambda functions can call APIs when needed. On the other hand, we have web services to be called as APIs by other systems and end users. This is mostly achieved via API Gateway.</p><p>A typical service connects to the system by subscribing to events needed to fulfill its purpose. Those inbound SNS events are then ingested via SQS allowing for orderly data processing and DLQs to handle errors. Outgoing results are pushed back to the system in form of events to a SNS topics.</p><p>As <a href="https://medium.com/u/75f5e85a1190">Marko Savic</a> has <a href="https://medium.com/homegate-engineering/replaying-events-using-aws-s3-3cc84242025e">pointed out</a>, we’re also utilizing S3 to store and replay those events if needed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LYNe9l9cm6cyNoAb36Ujqw.png" /><figcaption>Basic AWS Lambda function connected via SNS</figcaption></figure><p>If we would replace the SNS topic by streams on the EventBridge we would initially gain the same functionality by utilizing SQS as well. However, EventBridge has a different focus. Let’s look at the differences in general specs first.</p><h3><strong>Amazon SNS Specs</strong></h3><p>(not to be confused with our own Homegate “Search Notification Service” utilizing Amazon SNS.)</p><ul><li><em>Amazon SNS is a messaging bus based on publish/subscribe pattern</em></li><li><em>lots of consumers can subscribe (up to </em><strong><em>12.500.000 subscribers</em></strong><em>)</em></li><li><em>is comparably low latency (</em><strong><em>median latency: 25 ms</em></strong><em>)</em></li><li><em>listeners can define filters based on topics (and message attributes, but not on payload content)</em></li><li><em>ordering: </em><strong><em>unordered</em></strong><em> — but can be configured as FIFO (throughput will then be reduced)</em></li><li><em>security via: </em><a href="https://aws.amazon.com/kms/"><em>AWS KMS (Key Management Service)</em></a><em> &amp; server side encryption</em></li><li><em>message size: up to </em><strong><em>256 kB</em></strong></li><li><strong><em>USD 0.50 / 1 M messages</em></strong></li></ul><h3><strong>Amazon EventBridge</strong> Specs</h3><p>Is intended to connect micro services based on different AWS Products as well as external web services e.g. via API Gateways or AWS Lambda.</p><ul><li><em>default event bus already connected to ca. 100 AWS services</em></li><li><em>target services define filter rules to connect to events (max. </em><strong><em>5 targets</em></strong><em> per rule | max. </em><strong><em>100 rules</em></strong><em> per region)</em></li><li><em>filtering based on pattern matching in entire content (payload in JSON key detail)</em></li><li><strong><em>median latency: 560 ms</em></strong></li><li><em>ordering: </em><strong><em>unordered</em></strong></li><li><em>payload size: up to </em><strong><em>256 kB</em></strong><em> (ingestion of max. 10 events per message)</em></li><li><em>filtering/security via input transformer</em></li><li><strong><em>1 US$ / 1M events</em></strong></li></ul><p>Even though those general specs are quite different in some regards (such as latency and pricing); both services match all our of requirements for most services. But the specs are not really showing the biggest difference between the two.</p><h3><strong>The Heritage of EventBridge</strong></h3><p>EventBridge was first introduced as “<a href="https://docs.aws.amazon.com/AmazonCloudWatch/latest/events/WhatIsCloudWatchEvents.html">CloudWatch Events</a>” which was intended to allow monitoring based on log output. This is the reason for its filtering and pattern matching capabilities. It also brought with it a way of registering and discovering different EventBridge schemas. As it evolved the events of most existing AWS services where added to the schema registry.</p><p>Each SNS topics can be imagined as single message bus allowing the publication and subscription of messages by each micro service component (such as a Lambda or a Step Functions). Those messages are then concerning one specific, well: “Topic”.</p><p>On the other hand EventBridge is using one central default event bus <a href="https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-cross-account.html">per account</a>. Therefore it’s important to manage and filter the different types of events.</p><h3><strong>Schema discovery</strong></h3><p>We think the idea of a central schema repository for all events generated by our system is great. It automatically provides documentation and and gives an overview of the available events.</p><p>However, in order to make use of the schema repository the services need to interface it by generating and publishing them. We looked into the existing integration in the AWS console UI and especially the way to generate and push the schemas automatically to the registry.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Wr7T3mbmZDYoHApPX6_00Q.png" /><figcaption>AWS console UI of the EventBridge schema for AWSs own S3 object created event</figcaption></figure><p>The effort of generating and publishing the schemas (e.g. via CDK) turned out to be quite high and the UI for managing our schemas (in the AWS console) seemed to be very basic and missing features.</p><p>After all we where just planning to keep our own services to communicate with each other. We are also not using EventBridge extensively to connect to AWS services. Therefore the initiative to use it for new services has stalled and it seems unlikely that we will refactor existing services from using SNS to EventBridge for the time being.</p><h3>Conclusion</h3><p>At the current state EventBridge seems to be the perfect choice when allowing external services on AWS to connect deeply into a system.</p><p>The right question to ask when deciding wether to go with EventBridge seems not to be:</p><blockquote>“What can we gain by switching our SNS based service glue to EventBridge?”.</blockquote><p>It is rather:</p><blockquote>“Do we have (external) services that need to integrate with our events and that will profit greatly by the schema discovery?”.</blockquote><p>But for us, we will keep an eye on it (and the schema registry) and keep on using SNS as the glue of our own building blocks for the time being.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2d86b5919c22" width="1" height="1" alt=""><hr><p><a href="https://medium.com/smg-real-estate/micro-service-events-aws-eventbridge-vs-sns-2d86b5919c22">Micro Service Events (AWS EventBridge vs. SNS)</a> was originally published in <a href="https://medium.com/smg-real-estate">SMG Real Estate Engineering Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Configure renovate bot for npm at Homegate]]></title>
            <link>https://medium.com/smg-real-estate/configure-renovate-bot-for-npm-at-homegate-a6ae11b56ce3?source=rss----2186e5b9bd8f---4</link>
            <guid isPermaLink="false">https://medium.com/p/a6ae11b56ce3</guid>
            <category><![CDATA[renovate-bot]]></category>
            <category><![CDATA[npm]]></category>
            <category><![CDATA[dependencies]]></category>
            <category><![CDATA[automatic-deployment]]></category>
            <category><![CDATA[nodejs]]></category>
            <dc:creator><![CDATA[Khanh Nguyen]]></dc:creator>
            <pubDate>Mon, 23 May 2022 12:29:19 GMT</pubDate>
            <atom:updated>2022-05-23T12:29:19.141Z</atom:updated>
            <content:encoded><![CDATA[<p><a href="https://github.com/renovatebot/renovate">Renovate</a> bot is an automated dependency upgrade for npm packages. It supports many services such as gitlab &amp; github.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CBT2uuPCfeIsvkD-qcnacQ.png" /><figcaption>Homegate repositories structure</figcaption></figure><p>At Homegate, we have over <strong>200 repositories</strong> in micro-services architecture. Keeping services up to date is quite a big overhead. However, the benefits are obvious:</p><ul><li>Bugfixes</li><li>New features</li><li>Security issues / less vulnerabilities</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lp6hAMEV3Q_MHLHnIHlITA.png" /><figcaption>Evaluation session result from our engineering team</figcaption></figure><h3>The standard configuration</h3><p>We try to make process as automated and standardised as possible. The configuration will automatically create MRs for minor and major upgrade in our gitlab repos and auto-merge them.</p><p>It is necessary to allow merge without approval in these repositories. Below you’ll see our standard configuration with the following features:</p><ul><li>Schedule MR creations every hour on weekdays</li><li>Limits apply (concurrent MRs / Rate of creation)</li><li>Smart bundling can be configured, such as having all linters together or all of our shared frontend library related MRs together</li><li>Auto-merge for minor and dev-dependencies</li><li>Separation of major upgrade (requiring review) vs. minor upgrade (auto-merged)</li><li>Configuration for custom npm host</li><li>Each project has <em>renovate.json</em> that extends the standard <em>homegate.json</em> with custom config if necessary</li></ul><pre>{<br>  &quot;$schema&quot;: &quot;<a href="https://docs.renovatebot.com/renovate-schema.json">https://docs.renovatebot.com/renovate-schema.json</a>&quot;,<br>  &quot;extends&quot;: [<br>    &quot;config:base&quot;,<br>    &quot;:prHourlyLimitNone&quot;,<br>    &quot;:automergeLinters&quot;,<br>    &quot;:automergeTesters&quot;,<br>    &quot;:automergeTypes&quot;,<br>    &quot;:automergeMinor&quot;,<br>    &quot;group:definitelyTyped&quot;,<br>    &quot;group:postcss&quot;,<br>    &quot;group:linters&quot;,<br>    &quot;:rebaseStalePrs&quot;,<br>    &quot;:maintainLockFilesMonthly&quot;<br>  ],<br>  &quot;prConcurrentLimit&quot;: 2,<br>  &quot;schedule&quot;: [<br>    &quot;after 6am and before 5pm every weekday&quot;<br>  ],<br>  &quot;ignoreDeps&quot;: [&quot;node-sass&quot;,<br>    &quot;stylelint&quot;, &quot;stylelint-scss&quot;, &quot;stylelint-config-standard&quot;, &quot;stylelint-config-css-modules&quot;],<br>  &quot;packageRules&quot;: [<br>    {<br>      &quot;matchPackagePatterns&quot;: [&quot;vue&quot;, &quot;vue-router&quot;, &quot;vuex&quot;],<br>      &quot;matchUpdateTypes&quot;: [&quot;major&quot;],<br>      &quot;enabled&quot;: false<br>    },<br>    {<br>      &quot;matchPackagePatterns&quot;: [&quot;^<a href="http://twitter.com/aws">@aws</a>-cdk&quot;, &quot;aws-cdk&quot;],<br>      &quot;matchUpdateTypes&quot;: [&quot;major&quot;],<br>      &quot;enabled&quot;: false<br>    },<br>    {<br>      &quot;depTypeList&quot;: [&quot;devDependencies&quot;],<br>      &quot;automerge&quot;: true<br>    },<br>    {<br>      &quot;packageNames&quot;: [&quot;aws-sdk&quot;],<br>      &quot;extends&quot;: [&quot;schedule:weekly&quot;]<br>    },<br>    {<br>      &quot;sourceUrlPrefixes&quot;: [&quot;<a href="https://github.com/getndazn/dazn-lambda-powertools">https://github.com/getndazn/dazn-lambda-powertools</a>&quot;],<br>      &quot;groupName&quot;: &quot;Dazn lambda powertools packages&quot;<br>    },<br>    {<br>      &quot;sourceUrlPrefixes&quot;: [&quot;<a href="https://github.com/middyjs/middy">https://github.com/middyjs/middy</a>&quot;],<br>      &quot;groupName&quot;: &quot;Middy middleware packages&quot;<br>    },<br>    {<br>      &quot;sourceUrlPrefixes&quot;: [&quot;<a href="https://gitlab.com/homegate/projects/npm-packages/project-configs">https://gitlab.com/homegate/projects/npm-packages/project-configs</a>&quot;],<br>      &quot;groupName&quot;: &quot;<a href="http://twitter.com/homegate">@homegate</a> project configs packages&quot;<br>    },<br>    {<br>      &quot;sourceUrlPrefixes&quot;: [&quot;<a href="https://github.com/Turfjs/turf">https://github.com/Turfjs/turf</a>&quot;],<br>      &quot;groupName&quot;: &quot;TurfJs packages&quot;<br>    }<br>  ],<br>  &quot;addLabels&quot;: [&quot;renovate-bot&quot;],<br>  &quot;hostRules&quot;: [<br>    {<br>      &quot;hostType&quot;: &quot;npm&quot;,<br>      &quot;matchHost&quot;: &quot;homegate.jfrog.io/homegate/api/npm/homegate-npm&quot;,<br>      &quot;token&quot;: &quot;{{ secrets.JFROG_TOKEN }}&quot;<br>    }<br>  ],<br>  &quot;npmrc&quot;: &quot;<a href="http://twitter.com/homegate">@homegate</a>:registry=<a href="https://homegate.jfrog.io/homegate/api/npm/homegate-npm//n//homegate.jfrog.io/homegate/api/npm/homegate-npm/:_authToken=%7B%7B">https://homegate.jfrog.io/homegate/api/npm/homegate-npm/\n//homegate.jfrog.io/homegate/api/npm/homegate-npm/:_authToken={{</a> secrets.JFROG_TOKEN }}&quot;<br>}</pre><h3>Team responsibility</h3><p>It is important that each team allocate time to maintain and upgrade their libraries. We setup that renovate bot is mandatory in our services and even with the help with automation, team must take care of the status of what the bots are doing.</p><p>Most MRs have a team label added by the bot, this allows to query it. For example</p><blockquote><a href="https://gitlab.com/groups/homegate/projects/-/merge_requests?scope=all&amp;utf8=%E2%9C%93&amp;state=opened&amp;label_name[]=Rubber-Ducks">https://gitlab.com/groups/homegate/projects/-/merge_requests?scope=all&amp;utf8=%E2%9C%93&amp;state=opened&amp;label_name[]=Rubber-Ducks</a></blockquote><p>Sometimes, renovate MRs fail and need manual intervention. Here are some of the most usual issues and corresponding remediation:</p><ul><li>Flaky tests → retry job/pipeline</li><li>Rebase has conflict → manual rebase</li><li>Breaking change in MR that needs some update in codebase: <br>- Small changes → just do it<br>- Larger changes → create a ticket and close MR</li></ul><h3>Appendix: Custom configuration and process</h3><p>Our team, Honey Badgers, in Homegate owns the insertion-funnel which we found the standard process didn’t work out for us due to two main reasons:</p><ul><li>Front-end input heavy service -&gt; may bring breaking changes &amp; UI bugs especially</li><li>Many teams working on it in parallel -&gt; when working on a MR, continuous rebasing is required and we are in a reactive mode on fixing a problematic auto-upgrade (e.g. cypress) that may block other pipelines, which leads to unplanned distraction in case of issues.</li></ul><p><strong>We started to employ </strong>an extended configuration and process from the standard one:</p><ul><li>2 MRs on renovate (non-major &amp; major) for each project</li><li>Automatic generation of a ticket on JIRA board each week. The ticket is being picked up by the team and and MRs merged manually each week</li></ul><pre>{<br>  &quot;$schema&quot;: &quot;<a href="https://docs.renovatebot.com/renovate-schema.json">https://docs.renovatebot.com/renovate-schema.json</a>&quot;,<br>  &quot;extends&quot;: [<br>    &quot;gitlab&gt;homegate/projects/hg-renovate-bot:homegate&quot;,<br>    &quot;gitlab&gt;homegate/projects/hg-renovate-bot:label_honey-badgers&quot;<br>  ],<br>  &quot;timezone&quot;: &quot;Europe/Zurich&quot;,<br>  &quot;schedule&quot;: [&quot;before 8am on Monday&quot;],<br>  &quot;prConcurrentLimit&quot;: 10,<br>  &quot;packageRules&quot;: [<br>    {<br>      &quot;groupName&quot;: &quot;bot: devDependencies (non-major)&quot;,<br>      &quot;matchDepTypes&quot;: [&quot;devDependencies&quot;],<br>      &quot;matchUpdateTypes&quot;: [&quot;minor&quot;, &quot;patch&quot;, &quot;pin&quot;, &quot;digest&quot;],<br>      &quot;automerge&quot;: false<br>    },<br>    {<br>      &quot;groupName&quot;: &quot;bot: dependencies (non-major)&quot;,<br>      &quot;matchDepTypes&quot;: [&quot;dependencies&quot;],<br>      &quot;matchUpdateTypes&quot;: [&quot;minor&quot;, &quot;patch&quot;, &quot;pin&quot;, &quot;digest&quot;],<br>      &quot;automerge&quot;: false<br>    },<br>    {<br>      &quot;groupName&quot;: &quot;bot: devDependencies&quot;,<br>      &quot;matchDepTypes&quot;: [&quot;devDependencies&quot;],<br>      &quot;matchUpdateTypes&quot;: [&quot;major&quot;],<br>      &quot;automerge&quot;: false<br>    },<br>    {<br>      &quot;groupName&quot;: &quot;bot: dependencies&quot;,<br>      &quot;matchDepTypes&quot;: [&quot;dependencies&quot;],<br>      &quot;matchUpdateTypes&quot;: [&quot;major&quot;],<br>      &quot;automerge&quot;: false<br>    },<br>    {<br>      &quot;groupName&quot;: &quot;bot: serverless eco-system&quot;,<br>      &quot;matchPackageNames&quot;: [<br>        &quot;serverless&quot;,<br>        &quot;serverless-dotenv-plugin&quot;,<br>        &quot;serverless-finch&quot;<br>      ],<br>      &quot;automerge&quot;: false<br>    },<br>    {<br>      &quot;groupName&quot;: &quot;bot: disabled dependencies&quot;,<br>      &quot;matchPackageNames&quot;: [<br>        &quot;<a href="http://twitter.com/homegate/domus">@homegate/domus</a>&quot;,<br>        &quot;<a href="http://twitter.com/types/googlemaps">@types/googlemaps</a>&quot;,<br>        &quot;sass-loader&quot;,<br>        &quot;mock-xmlhttprequest&quot;,<br>        &quot;<a href="http://twitter.com/vue/test-utils">@vue/test-utils</a>&quot;,<br>        &quot;<a href="http://twitter.com/homegate/hg-accounts-client">@homegate/hg-accounts-client</a>&quot;,<br>        &quot;vue-tel-input&quot;,<br>        &quot;eslint&quot;,<br>        &quot;file-type&quot;,<br>        &quot;ts-jest&quot;,<br>        &quot;stylelint-config-css-modules&quot;<br>      ],<br>      &quot;enabled&quot;: false<br>    }<br>  ]<br>}</pre><p><strong>Results: </strong>This provides control on upgrades while having benefits of a semi-automated process. We spend around 4 hours a month to keep up dependencies. Here are the main takeaways for us:</p><ul><li>Optimize time to work or react on upgrading libs issues</li><li>Reduce number of MRs created</li><li>Have a semi-automated process to keep libs upgraded</li><li>Avoid risks of broken pipelines or unwanted feature changes by having a manual merge process</li></ul><h3>Credits</h3><p>Homegate engineers.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a6ae11b56ce3" width="1" height="1" alt=""><hr><p><a href="https://medium.com/smg-real-estate/configure-renovate-bot-for-npm-at-homegate-a6ae11b56ce3">Configure renovate bot for npm at Homegate</a> was originally published in <a href="https://medium.com/smg-real-estate">SMG Real Estate Engineering Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Coroutines on Android]]></title>
            <link>https://medium.com/smg-real-estate/coroutines-on-android-d3e3413e6aa7?source=rss----2186e5b9bd8f---4</link>
            <guid isPermaLink="false">https://medium.com/p/d3e3413e6aa7</guid>
            <category><![CDATA[kotlin-coroutines]]></category>
            <category><![CDATA[coroutine-testing]]></category>
            <category><![CDATA[android-app-development]]></category>
            <category><![CDATA[android]]></category>
            <category><![CDATA[android-apps]]></category>
            <dc:creator><![CDATA[Stevan Milovanovic]]></dc:creator>
            <pubDate>Fri, 20 May 2022 12:36:23 GMT</pubDate>
            <atom:updated>2022-05-26T06:50:36.952Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*05FDJAgFi7IYnFFyGZ3YNA.png" /><figcaption><a href="https://miro.medium.com/max/1400/1*6iokPre2Cp3ujsGZqzBDWA.png">Coroutines on Android</a></figcaption></figure><p>In this article I’ll try to explain why coroutines are useful and why would you want to use them in your project. After we go through the most important concepts of coroutines, I’ll show you how I used coroutines to implement networking and background processing in the example project.</p><p>First of all, you might ask yourself, why we want to use coroutines over threads? Main problem with threads is that they are resource intensive, meaning it takes a lot of resources to start a thread, stop a thread. Meanwhile, coroutines are lightweight threads, since they use thread pools. Another benefit of coroutines is that they greatly simplify asynchronous code. Callbacks and synchronisation are very easy to use. In fact, they make parallel programming look very much like sequential programming. Coroutines can be paused and resumed at any time, on a number of threads. And lastly, since coroutines are based on a few fairly easy to grasp concepts, their syntax is simple and easy to use. Here are the main concepts we need to explain about coroutines:</p><p><strong>Scope</strong></p><p>Coroutine scope, as its name says, defines a scope for new coroutines. Every <strong>coroutine builder</strong> (like <a href="https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/launch.html">launch</a> and <a href="https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/async.html">async</a>) is an extension on <a href="https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-coroutine-scope/index.html">CoroutineScope</a> and inherits its <a href="https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-coroutine-scope/coroutine-context.html">coroutineContext</a> to automatically propagate all its elements and cancellation.</p><p><strong>Context</strong></p><p>Coroutine context represents the context of its scope. Context is encapsulated by the scope and used for implementation of coroutine builders that are extensions on the scope. Scope provides a context in which the coroutine runs (state of the coroutine which provides variables, functionality of the coroutine etc.).</p><p><strong>Suspending functions</strong></p><p>Suspending functions are functions that can be run in a coroutine. They make callbacks seamless. They can be run in a coroutine (can be suspended) and that is why they can provide functionalities which have to be run in parallel.</p><p><strong>Job</strong></p><p><a href="https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-job/index.html">Job</a> is a handle on that coroutine (on the piece of code which runs in the background). A <a href="https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/launch.html">launch</a>() call returns a <a href="https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-job/index.html">Job</a> which allows us to manipulate the coroutine lifecycle. Jobs live in the hierarchy of other Jobs both as parents or children.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IZxjr5Qnq56z1F5J3anhBw.png" /><figcaption>Job hierarchy</figcaption></figure><p>A job has the following states:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fpELIegoYgw305DcH4jM6g.png" /><figcaption>Job states</figcaption></figure><p>Most important methods of a job are cancel method which cancel it and join, which joins this job to the current thread. If a job is cancelled, all its parents and children will be cancelled too.</p><p><strong>Dispatcher</strong></p><p>Dispatcher determines which thread or thread pool the coroutine runs on. Different dispatchers are available depending on the task specificity. For example, default dispatcher is designed for very CPU intensive processing tasks. Common dispatchers are main — for updating UI, default — for data/image processing (CPU intensive work), IO — designed for network communication or reading/writing files and unconfined dispatcher which starts the coroutine in the inherited dispatcher that called it. If you simply want a dispatcher which will create a new thread for you, you can use newSingleThreadContext(“MyThread”) which will force creation of a new thread. This shouldn’t be used that often since threads are very expensive and coroutines are lightweight so try to use coroutines as they are intended and allow dispatcher to move around from thread to thread.</p><p><strong>Coroutine builders</strong></p><p>There are two main coroutine builders, <a href="https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/launch.html">launch</a> and <a href="https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/async.html">async</a>. As it is described earlier, launch returns a job which is placed in jobs hierarchy. Async is another way to start the coroutine. It is just like a launch method, except it returns a result in the form of a Deferred. When we need the value, we can call await() (it’s a blocking call). If the value is available, it will return immediately and if not, it will pause the thread until it is.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/767dafbac35eda6760ca579996119268/href">https://medium.com/media/767dafbac35eda6760ca579996119268/href</a></iframe><p><strong>Suspending functions</strong></p><p>Coroutines build upon regular functions by adding two new operations. In addition to <strong>invoke</strong> (or call) and <strong>return</strong>, coroutines add <strong>suspend</strong> and <strong>resume</strong>.</p><ul><li><strong>suspend</strong> — pause the execution of the current coroutine, saving all local variables</li><li><strong>resume</strong> — continue a suspended coroutine from the place it was paused</li></ul><p>A <strong>suspending function</strong> is just a regular <strong>Kotlin function</strong> with an additional <strong>suspend</strong> modifier which indicates that the <strong>function</strong> can <strong>suspend</strong> the execution of a coroutine.</p><pre>suspend fun getWeather(cityId: Int): Forecast</pre><p>You can only call suspend functions from other suspend functions, or by using a coroutine builder like <a href="https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/launch.html">launch</a> to start a new coroutine.</p><p><strong>withContext</strong></p><p>A function that is very commonly used in coroutines is withContext(). It allows us to easily change the context or basically to easily switch between dispatchers. If we have let’s say, some heavy image processing, we would do it with default dispatcher. When the outcome is completed, we would want to switch to the main dispatcher in order to update the ui with the resulting image. The best way to do this is withContext function since it is very lightweight. A way to do this is:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/978ecf81746a16a7a2a21a1f03647c65/href">https://medium.com/media/978ecf81746a16a7a2a21a1f03647c65/href</a></iframe><p><strong>Exception handling</strong></p><p>Exception behaviour depends on the coroutine builder. We have two coroutine builders: launch and async. Launch generates a job and async generates deferred result. Since launch creates a job hierarchy, the exception will propagate throughout the hierarchy and will immediately cause job to fail. The way of handling exceptions is to use try catch block inside of the launch block or use structure called exception handler.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/bb3150c17279d8ac9059c90c856cead5/href">https://medium.com/media/bb3150c17279d8ac9059c90c856cead5/href</a></iframe><p>On the other side, handling exception with async method is slightly different. Async method doesn’t provide hierarchy, it just provides outcome. The exception is deferred, just like the result and it is not shown until is consumed. If we don’t consume the result, the exception won’t be thrown. So, in this case, await call should be wrapped with the try catch block.</p><p><strong>Example project</strong></p><p>In the example project, I’ll try to demonstrate how you can use coroutines with most commonly used library for network operations (<a href="https://square.github.io/retrofit/">Retrofit</a>) to pull the data from the remote server and process it locally. Project is hosted on the GitHub (<a href="https://github.com/stevan-milovanovic/coroutines">https://github.com/stevan-milovanovic/coroutines</a>) and it shows the weather forecast for the city in which the user is currently located.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/324/1*sWgpUL6DY73aQ1GO3-K0nA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/324/1*dXI5p-DLH6ajGySqk4nUzA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/324/1*9Is5x7IPu-XTz9H0rBzoeA.png" /><figcaption>weather forecast example app</figcaption></figure><p><strong>Retrofit and Coroutines</strong></p><p>Retrofit is a type-safe HTTP client for Android and Java. Since it provides <strong>built-in support for Kotlin suspend modifier on functions</strong>, we can easily define our interface:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/917d95384b21935ae30d3e1c86a7fb11/href">https://medium.com/media/917d95384b21935ae30d3e1c86a7fb11/href</a></iframe><p><strong>ViewModel and Coroutines</strong></p><p>A <a href="https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-coroutine-scope/">CoroutineScope</a> keeps track of all coroutines it creates. Therefore, if you cancel a scope, you cancel all coroutines it created. This is particularly important if you’re running coroutines in a <a href="https://developer.android.com/topic/libraries/architecture/viewmodel">ViewModel</a>.</p><p>If your ViewModel is getting destroyed, all the asynchronous work that it might be doing must be stopped. Otherwise, you’ll waste resources and potentially leaking memory. If you consider that certain asynchronous work should persist after ViewModel destruction, it is because it should be done in a lower layer of your app’s architecture. For context tied to the viewModel lifecycle, ViewModel extension is created called <a href="https://developer.android.com/topic/libraries/architecture/coroutines#viewmodelscope"><em>viewModelScope</em></a>. In this example, I am using viewModelScope to launch both network operation and local file processing, although I am using IO Dispatcher for execution.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/616d6f2a16af19206aa0348cc75beb53/href">https://medium.com/media/616d6f2a16af19206aa0348cc75beb53/href</a></iframe><p><strong>Testing Coroutines</strong></p><p>For testing coroutines, I made custom test rule named TestCoroutineRule so that everything can be executed on the same thread:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/8fa04824e0721a3c66bbb102911cedc0/href">https://medium.com/media/8fa04824e0721a3c66bbb102911cedc0/href</a></iframe><p>and this is how it is used for writing a test:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c5a567160c070f8c2272cd4802df9f30/href">https://medium.com/media/c5a567160c070f8c2272cd4802df9f30/href</a></iframe><p>Thanks for reading. Checkout this example in <a href="https://github.com/stevan-milovanovic/coroutines">GitHub</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d3e3413e6aa7" width="1" height="1" alt=""><hr><p><a href="https://medium.com/smg-real-estate/coroutines-on-android-d3e3413e6aa7">Coroutines on Android</a> was originally published in <a href="https://medium.com/smg-real-estate">SMG Real Estate Engineering Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Replaying events using AWS S3]]></title>
            <link>https://medium.com/smg-real-estate/replaying-events-using-aws-s3-3cc84242025e?source=rss----2186e5b9bd8f---4</link>
            <guid isPermaLink="false">https://medium.com/p/3cc84242025e</guid>
            <category><![CDATA[aws-lambda]]></category>
            <category><![CDATA[event-sourcing]]></category>
            <category><![CDATA[software-architecture]]></category>
            <category><![CDATA[aws-s3]]></category>
            <category><![CDATA[serverless]]></category>
            <dc:creator><![CDATA[Marko Savic]]></dc:creator>
            <pubDate>Fri, 08 Apr 2022 08:40:24 GMT</pubDate>
            <atom:updated>2022-04-08T09:40:53.098Z</atom:updated>
            <content:encoded><![CDATA[<p>In the world of <em>event-driven microservices </em>one of the important tasks is to have a mechanism to recover from failed events, as well as to have possibility to replay certain events. We at Homegate deal with <em>serverless</em> microservices. If, at any of these services, we encounter a failure during processing of an resource, we might need to recreate the last event or series of events for given resource.</p><h4>Usecase</h4><p>The central part of ingestion process is <em>AWS StepFunctions</em> that lives in a, so called, importer service. It is responsible for various enrichments and conditional actions on resource. The last step of the step function is sending an event to the <em>AWS SNS</em> which is consumed by many downstream services as you can see in the following diagram.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tnSgNjPhSIsxBbRfz-gJ-A.png" /><figcaption>Step functions as central part of ingestion process</figcaption></figure><p>Here, many open questions can be identified:</p><ul><li>How to deal with failure within AWS StepFunctions?</li><li>How to recreate SNS event if needed for downstream system?</li><li>Would it be possible to replay the whole step function?</li></ul><h4>Solution</h4><p>Possible answer to these questions would be to store events that goes through AWS StepFunctions into some kind of event storage, in Homegate case, we chose AWS S3. In order to support the whole history of events for given resource, S3 bucket versioning is used. This is quite handy for replaying series of events or even for debugging purposes. Here is a sample CloudFormation which enables that functionality:</p><pre>EventLogBucket:<br>  Type: &#39;AWS::S3::Bucket&#39;<br>  Properties:<br>    BucketName: &#39;sample-unnique-bucket-name&#39;<br>    AccessControl: Private<br>    PublicAccessBlockConfiguration:<br>      BlockPublicAcls: true<br>      BlockPublicPolicy: true<br>      IgnorePublicAcls: true<br>      RestrictPublicBuckets: true<br>    Tags:<br>      - Key: service<br>        Value: sample-service<br>      - Key: env<br>        Value: !FindInMap [ Stages, !Ref Stage , Env ]<br>    <strong>VersioningConfiguration:<br>      Status: Enabled</strong></pre><p>That way we can replay the whole step functions flow using last logged event for given resource. That functionality could be available from different places, like DLQ or via api endpoint. In the following diagram and the description below it will be explained how all these components communicate with each other.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qvXzBf2l31NQXmO8UilVTw.png" /><figcaption>S3 event log bucket as a central part of replay mechanism</figcaption></figure><p>As you can see, in this case, the first step in given <em>AWS StepFunctions</em> would be to store the event into corresponding S3 bucket. As a key for S3, resource id can be used. Here is the example of S3 structure with versioning enabled, where for the same object id we have all the history of events available:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cij2hYX7rdkD_Al_3Wc6kQ.png" /><figcaption>Object versioning in S3</figcaption></figure><p>In case of a failure simple DLQ could be used for the error handling. Here, different approaches would be possible like having an auto replay mechanism which will, on schedule, trigger lambda function to fetch messages from DLQ and trigger replay. If all conditions are satisfied (for example if error is retriable) given lambda will fetch latest event for given resource and trigger step function again.</p><p>Replay functionality is also available from outside of the service in case we need to repeat importing of certain resources. In order to have more flexibility, this is exposed over <em>AWS API Gateway </em>where consumer of the endpoint can provide various input params like multiple resources, or all the resources that belongs to a specific user, or if the new execution would need a special treatment (replaying series of events or have specific data manipulation). To support this flexibility, a <em>Replay SQS </em>is added between <em>Replay Handler Lambda </em>and <em>Replay Listener Lambda.</em></p><p>Finally, <em>Replay Listener</em> is responsible to fetch the events from <em>S3</em> and trigger <em>step functions</em> according to given input.</p><p>To conclude, in modern system architecture, having a possibility to replay events is a necessity and has many usecases. As we saw in the simplified real-life example from above, AWS S3 could be used as a simple, yet very effective helper tool to replay events and help to ensure consistency of intercommunicating microservices.</p><h4>References</h4><ul><li><a href="https://docs.aws.amazon.com/AmazonS3/latest/userguide/manage-versioning-examples.html">https://docs.aws.amazon.com/AmazonS3/latest/userguide/manage-versioning-examples.html</a></li><li><a href="https://aws.amazon.com/blogs/compute/using-amazon-sqs-dead-letter-queues-to-replay-messages/">https://aws.amazon.com/blogs/compute/using-amazon-sqs-dead-letter-queues-to-replay-messages/</a></li></ul><h4>See also</h4><ul><li><a href="https://martinfowler.com/eaaDev/EventSourcing.html">https://martinfowler.com/eaaDev/EventSourcing.html</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3cc84242025e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/smg-real-estate/replaying-events-using-aws-s3-3cc84242025e">Replaying events using AWS S3</a> was originally published in <a href="https://medium.com/smg-real-estate">SMG Real Estate Engineering Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AWS FIFO Queues with Message Groups for Atomic Processing at Scale]]></title>
            <link>https://medium.com/smg-real-estate/aws-fifo-queues-with-message-groups-for-atomic-processing-at-scale-dc70f8820b16?source=rss----2186e5b9bd8f---4</link>
            <guid isPermaLink="false">https://medium.com/p/dc70f8820b16</guid>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[sqs]]></category>
            <category><![CDATA[serverless]]></category>
            <category><![CDATA[fifo]]></category>
            <dc:creator><![CDATA[Nick De Cooman]]></dc:creator>
            <pubDate>Thu, 03 Mar 2022 09:55:58 GMT</pubDate>
            <atom:updated>2022-03-03T09:55:57.980Z</atom:updated>
            <content:encoded><![CDATA[<h4>How to process multiple objects in parallel with the guarantee that for each unique object, only a single instance is processed simultaneously</h4><p>In a distributed, event-driven architecture, many challenges arise when processing large sets of data. One of them is the need for scalability when handling event streams with a substantial throughput. The challenge becomes even more complicated when processing these events involves atomic operations. In this case, we cannot simply process multiple events in parallel.</p><p>As an example, let’s say that we are building a service that ingests listings into our platform. The ingestion process involves several steps and requires multiple API operations. As a result, the whole ingestion can take up to several seconds per listing to complete.</p><p>Now, here is where it gets tricky: a constraint of our service is that, while we are processing a listing, we cannot process other events for the same listing. The ingestion should be considered an atomic operation, and hence, only a single execution per listing can be performed simultaneously.</p><p>At <a href="https://www.homegate.ch/">Homegate</a>, we run most of our infrastructure on AWS in a serverless fashion. At the core of this, Lambda functions execute application logic. A common pattern is to use SNS topics to fan-out messages, and SQS for acting as a decoupling buffer between an SNS topic and some Lambda function.</p><p>For our ingester service, a logical architecture could look like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*o6i0i7H0QYLlLTaRxLo8Ow.jpeg" /></figure><p>Listing events are published to a dedicated SNS topic. An SQS queue subscribes to this topic and consumes the events. Here, the events are buffered until a Lambda instance removes them from the queue and processes the corresponding listing.</p><p>Under normal circumstances — in which the processing would not involve an atomic operation — this architecture would be a good fit. It would scale automatically relative to the number of Lambda instances that consume events from the queue, without much additional configuration.</p><p>However, in the case where we cannot process two events for a single listing at the same time, things become trickier. If we just allow multiple Lambda instances to consume events from our queue, a single listing might get processed twice exactly at the same time.</p><p>A simple yet effective solution could be to limit the number of Lambda instances that can consume events from the queue. If we only allow a single Lambda instance to remove events from the queue, only one listing can be processed at a time. Solving concurrency by simply avoiding it. 🤷🏻‍♂️</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LBp7eJ9PiCRrJvstfOpuyw.jpeg" /></figure><p>Compelling as it is, this strategy comes with a considerable performance cost. Since only one single listing can be processed at a time, it is easy to see that this creates a bottleneck. We need a better solution for services requiring a high throughput, like our ingester.</p><p><strong>Introducing AWS SQS FIFO queues</strong></p><p>AWS SQS FIFO queues are like regular SQS queues, but — as the name suggests — they also guarantee a first-in, first-out order. Additionally, they ensure that events are received exactly once (without duplicates), which is not the case with regular queues.</p><p>Perhaps a less known feature of FIFO queues is the ability to assign a message group id to each of the incoming events. Events with the same message group id belong to the same message group. Within this group, events are always processed one by one in a strict order relative to the message group.</p><p>Back to our ingester service, where we can take full advantage of message groups: by replacing the regular SQS queue with a FIFO queue, we can assign the listing id as the message group id for each incoming event. This ensures we process only a single event per listing simultaneously.</p><p>Given this guarantee, it is now safe to scale up the number of Lambda instances consuming the queue. As long as a Lambda instance is busy processing an event from a specific message group, no other Lambda instance can consume an event for the same message group. Since a message group represents a listing, this behavior is exactly what we need to make sure that only a single event per listing is processed at a time.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1C0EVyJVCIrlFE1W1TJ-QA.jpeg" /></figure><p>With more Lambda instances now active, we solved the earlier bottleneck and scaled our throughput relative to the number of active Lambda instances.</p><p><strong>Show me the code</strong></p><p>Transitioning from a regular SQS queue to a FIFO queue is straightforward. With CDK, all it takes is adding .fifo as a suffix to the name of the queue and setting fifo: true while initializing the queue.</p><pre>const fifoQueue = new Queue(this, &#39;IngestQueue.fifo&#39;, {</pre><pre>  fifo: true</pre><pre>})</pre><p>To assign a message group id to an event, SQS provides a MessageGroupId property when sending a message to the queue. In the example below, we use the listing id as the message group id.</p><pre>import AWS from &#39;aws-sdk&#39;;</pre><pre>const sqs = new AWS.SQS({ apiVersion: &#39;2012-11-05&#39; });<br></pre><pre>const listing = { id: &#39;784c6d44&#39;, ... }</pre><pre>const sqsParams = {</pre><pre>  MessageBody: JSON.stringify(listing),</pre><pre>  MessageGroupId: listing.id,</pre><pre>  QueueUrl: fifoQueueUrl</pre><pre>};<br></pre><pre>await sqs.sendMessage(sqsParams).promise()</pre><p>In conclusion, AWS FIFO queues are an excellent addition to the AWS toolset. They enhance messaging between applications in case the order of events is critical, or when duplicates can not be tolerated. In our example, we greatly benefited from the message group feature provided by FIFO queues. By using the listing id as the message group, we managed to process multiple listings in parallel, while at the same time having the guarantee that only a single Lambda instance per listing is running simultaneously.</p><h3>References</h3><ul><li><a href="https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/FIFO-queues.html">https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/FIFO-queues.html</a></li><li><a href="https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/partitions-and-data-distribution.html">https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/partitions-and-data-distribution.html</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dc70f8820b16" width="1" height="1" alt=""><hr><p><a href="https://medium.com/smg-real-estate/aws-fifo-queues-with-message-groups-for-atomic-processing-at-scale-dc70f8820b16">AWS FIFO Queues with Message Groups for Atomic Processing at Scale</a> was originally published in <a href="https://medium.com/smg-real-estate">SMG Real Estate Engineering Blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>