<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Mohammed Mostafa on Medium]]></title>
        <description><![CDATA[Stories by Mohammed Mostafa on Medium]]></description>
        <link>https://medium.com/@elrefai99?source=rss-1cb366e94de2------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*wu7gK4Qmg5su9NldUZYWVw.jpeg</url>
            <title>Stories by Mohammed Mostafa on Medium</title>
            <link>https://medium.com/@elrefai99?source=rss-1cb366e94de2------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 05:39:54 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@elrefai99/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[I Built a Go Proxy for HTTP, TCP, and UDP]]></title>
            <link>https://elrefai99.medium.com/i-built-a-go-proxy-for-http-tcp-and-udp-c9e18226d42f?source=rss-1cb366e94de2------2</link>
            <guid isPermaLink="false">https://medium.com/p/c9e18226d42f</guid>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[https]]></category>
            <category><![CDATA[tcp]]></category>
            <category><![CDATA[udp]]></category>
            <category><![CDATA[docker]]></category>
            <dc:creator><![CDATA[Mohammed Mostafa]]></dc:creator>
            <pubDate>Tue, 17 Nov 2026 10:49:22 GMT</pubDate>
            <atom:updated>2026-11-17T10:49:48.646Z</atom:updated>
            <content:encoded><![CDATA[<p><em>*Leoxy started with a familiar reverse-proxy problem: one machine, several services, and too many places to configure how traffic reaches them.*</em></p><p>Most reverse proxies are introduced with a simple diagram: a request comes in, a path is matched, and traffic is forwarded to an application. That model works well for web services. It becomes less complete when the same deployment also needs to expose a TCP service or a UDP listener.</p><p>I built [Leoxy](https://leoxy.vercel.app/) to make those setups possible through one small Go service. It routes HTTP requests by path and can run separate TCP and UDP listeners. The interesting part wasn’t just forwarding bytes. It was making the configuration, security boundaries, health signals, and deployment behavior clear enough to operate.</p><p><strong>## Start with the shape of the deployment</strong></p><p>Leoxy uses YAML to describe its HTTP listener and upstream routes. A minimal route looks like this:</p><p>```yaml</p><p>server:</p><p>host: “127.0.0.1”</p><p>port: “8080”</p><p>upstream:</p><p>- name: “Backend”</p><p>protocol: “http”</p><p>path: “/api”</p><p>server_url: “http://localhost:9000&quot;</p><p>```</p><p>This sends requests under `/api` to the backend. A route can target one server or a list of servers, and `/` can be used as a catch-all. HTTP paths must be unique. Paths such as `/health`, `/metrics`, and `/leoxy` are reserved for Leoxy’s own endpoints.</p><p>TCP and UDP need a different shape. They don’t have HTTP paths to match, so each stream entry declares a listening address and a `host:port` destination. That distinction keeps the config honest: HTTP routes are selected by path, while stream traffic enters through its own listener.</p><p><strong>## Don’t let protocol names promise too much</strong></p><p>Leoxy accepts HTTP-family labels such as `http`, `https`, `http2`, `graphql`, and `websocket`. They all use the same HTTP reverse-proxy handler. Choosing a label does not create a separate listener or a different protocol implementation. TCP and UDP are the separate stream modes.</p><p>This is easy to gloss over when designing a config format. A label can look like a promise. If `websocket` appears next to `tcp`, an operator might reasonably expect them to behave as separate transports. I chose to document that the HTTP-family labels share one handler and keep TCP/UDP modeling distinct.</p><p>The tradeoff is that documentation has to be precise. The config remains familiar, but the docs must explain the runtime behavior instead of letting a label speak for itself.</p><p><strong>## Security is a question of scope</strong></p><p>A proxy is a useful place to enforce policy. Leoxy supports shared HTTP security settings and route-level overrides, including:</p><p>- Per-client and route-level rate limits, plus a shared global HTTP limit.</p><p>- API key checks and HS256 JWT verification.</p><p>- CIDR allow and deny rules.</p><p>- HTTP method and path restrictions.</p><p>- Request body limits and trusted proxy ranges.</p><p>- TLS and mutual TLS on the HTTP listener.</p><p>The global HTTP limiter can use local process memory or Redis. Local memory keeps a single-instance deployment simple. Redis allows multiple instances to share a limit, but adds an external dependency. The config also makes Redis failure behavior explicit: fail closed, or fall back to a local limiter.</p><p>The stream policies are narrower. TCP and UDP support CIDR rules and per-client connection limits. HTTP authentication, HTTP rate limits, and the HTTP TLS settings do not apply to stream traffic. Saying that plainly matters more than making the security section look comprehensive. An operator needs to know which controls actually protect which traffic.</p><p><strong>## A process can be alive while an upstream is down</strong></p><p>Operational checks are another place where names matter. Leoxy exposes separate endpoints for different questions:</p><p>- `/health/live` reports whether the process is running.</p><p>- `/health/ready` checks the configured HTTP upstreams.</p><p>- `/metrics` exposes request and response metrics, along with watcher samples.</p><p>- `/ping` provides a lightweight ping response.</p><p>- `/leoxy` serves the bundled information page.</p><p>Liveness is not readiness. A proxy process can be running while an application behind it is unavailable. Keeping the checks separate lets a container runtime test whether Leoxy itself is alive without claiming that every dependency is healthy.</p><p>There is also an optional resource watcher. When enabled with a positive memory threshold, it samples Go runtime resources and can write a report and heap profile when the threshold is reached. Profiling endpoints are disabled unless explicitly enabled, so diagnostics aren’t exposed by default.</p><p><strong>## Make the first run safe and the next run manageable</strong></p><p>Leoxy’s CLI can run in the foreground or manage a background process with `start`, `restart`, and `stop`. The ` — config` flag selects a YAML file, while ` — data-dir` sets the working directory for relative paths and runtime data.</p><p>If the default config file is missing, Leoxy creates a small one. It doesn’t silently copy a sample deployment with application routes. That means the operator needs to add upstreams before routing real traffic, but it avoids a sample route being mistaken for a production setup.</p><p>Leoxy can run as a binary or from its published Docker image. The container listens on port 80 and reads `/app/leoxy/leoxy_config.yaml`. Mounting a local directory there keeps the config and runtime files outside the container lifecycle.</p><p>```sh</p><p>docker pull mohamed0699/leoxy:latest</p><p>docker run — rm -p 8080:80 -v “$PWD/leoxy:/app/leoxy” mohamed0699/leoxy:latest</p><p>```</p><p>One detail catches people when a proxy runs in Docker: `localhost` inside the container refers to the container itself. To reach a host service or another container, Leoxy needs an address reachable from that container, such as a Compose service name or `host.docker.internal` where supported.</p><p>The image includes a liveness check at `/health/live`. The mounted config still needs the real upstreams before application traffic can be routed.</p><p><strong>## Documentation is part of the product</strong></p><p>Configuration-heavy software has another interface beyond its CLI and code: the file operators edit when something goes wrong. Leoxy’s docs cover defaults, field meanings, environment overrides, reserved paths, timeout behavior, and which security settings apply to each protocol.</p><p>The reference also calls out settings that exist in the schema but are not enforced by the current runtime. That may be less exciting than adding another feature, but it prevents a field name from being mistaken for working behavior. Good docs should describe what the software does today, including its boundaries.</p><p><strong>## A small proxy with explicit boundaries</strong></p><p>Leoxy combines path-based HTTP routing with separate TCP and UDP forwarding in a configurable Go service. Around that core, it provides scoped security controls, health checks, metrics, resource monitoring, a lifecycle-aware CLI, and a Docker distribution.</p><p>I’m not positioning it as a benchmark winner or a replacement for every edge proxy. The point is a compact tool whose config and operational behavior are understandable: what gets routed, which policy applies, what a health check means, and where the runtime keeps its files.</p><p>Explore the [live project](https://leoxy.vercel.app/), browse the [source on GitHub](https://github.com/LX-2v/Leoxy), or find the image on [Docker Hub](https://hub.docker.com/r/mohamed0699/leoxy).</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c9e18226d42f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Leoxy: One Go Proxy for HTTP, TCP, and UDP]]></title>
            <link>https://elrefai99.medium.com/leoxy-one-go-proxy-for-http-tcp-and-udp-4f2b6a0002bd?source=rss-1cb366e94de2------2</link>
            <guid isPermaLink="false">https://medium.com/p/4f2b6a0002bd</guid>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[https]]></category>
            <category><![CDATA[proxy]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[tcp]]></category>
            <dc:creator><![CDATA[Mohammed Mostafa]]></dc:creator>
            <pubDate>Tue, 17 Nov 2026 10:44:33 GMT</pubDate>
            <atom:updated>2026-11-17T10:45:42.699Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>Leoxy: One Go Proxy for HTTP, TCP, and UDP</strong></p><p>Some reverse proxies are built around one job: accept an HTTP request, match a path, and forward it to an application. That covers a lot of deployments, but it leaves a gap as soon as the same host also needs to expose a game server, a telemetry listener, or another service that does not speak HTTP.</p><p>I built [Leoxy](https://leoxy.vercel.app/) as a small Go proxy that can handle both sides of that boundary. It routes HTTP traffic by path and can run independent TCP and UDP listeners. The goal was to keep the setup declarative and the operational behavior explicit, without making the configuration pretend that all protocols work the same way.</p><p><strong>## A config file should describe the deployment</strong></p><p>Leoxy uses YAML to define the HTTP listener, upstream routes, stream listeners, timeouts, security policies, and optional resource monitoring. A minimal HTTP route looks like this:</p><p>```yaml</p><p>server:</p><p>host: “127.0.0.1”</p><p>port: “8080”</p><p>upstream:</p><p>- name: “Backend”</p><p>protocol: “http”</p><p>path: “/api”</p><p>server_url: “http://localhost:9000&quot;</p><p>```</p><p>The route sends requests under `/api` to the backend. A route can target one server or use a list of servers. A `/` route can act as a catch-all. HTTP paths must be unique, and Leoxy reserves paths such as `/health`, `/metrics`, and `/leoxy` for its own endpoints.</p><p>TCP and UDP entries describe a different shape: each one has its own listening address and a `host:port` target. There is no URL path to match, so the configuration makes the listener and destination explicit. This distinction is small in YAML, but useful operationally: the file shows which ports are exposed and where each stream is sent.</p><p><strong>## Keep protocol behavior honest</strong></p><p>The configuration accepts several HTTP-family labels, including `http`, `https`, `http2`, `graphql`, and `websocket`. They use the same HTTP reverse-proxy handler; choosing one of these labels does not start a separate listener or create a distinct protocol implementation. TCP and UDP are the separate stream modes.</p><p>That detail matters because configuration names can imply capabilities that a runtime does not provide. I chose to document the shared HTTP behavior and model TCP/UDP separately, rather than suggest that every label changes how traffic is handled. The tradeoff is that operators need to understand the distinction between a protocol label and a different transport path through the application.</p><p><strong>## Put security where it can be understood</strong></p><p>A proxy sits at a useful enforcement point, but a long list of controls is only helpful when their scope is clear. Leoxy supports shared HTTP policy and route-specific policy for rate limits, API key checks, HS256 JWT verification, CIDR allow and deny rules, method and path restrictions, body limits, trusted proxy ranges, and TLS. Mutual TLS is available when the listener certificate, key, and client CA are configured.</p><p>There is also a shared global HTTP rate limiter. It can use local process memory or Redis when multiple instances need a shared limit. Redis introduces an external dependency, so the configuration makes the failure behavior explicit: deployments can choose whether a Redis error should fail closed or fall back to a local limiter.</p><p>Stream listeners have a narrower policy surface. TCP and UDP support CIDR access rules and per-client connection limits. HTTP authentication, HTTP rate limits, and the HTTP TLS settings do not apply to stream traffic. Keeping that boundary visible is safer than implying one security block has identical meaning across every transport.</p><p><strong>## Separate “alive” from “ready”</strong></p><p>Leoxy exposes several endpoints for operations:</p><p>- `/health/live` reports whether the process is running.</p><p>- `/health/ready` checks readiness of configured HTTP upstreams.</p><p>- `/metrics` exposes request and response metrics, along with watcher samples.</p><p>- `/ping` provides a lightweight ping response.</p><p>- `/leoxy` serves the bundled information page.</p><p>Liveness and readiness answer different questions. A running proxy process can be alive while an upstream is unavailable. Keeping those signals separate lets a container runtime check the process without confusing that check with application dependency health.</p><p>An optional resource watcher samples Go runtime resources when enabled and a positive memory threshold is configured. It can write reports and heap profiles when the threshold is reached, with a cooldown between reports. Go profiling endpoints are separately disabled unless explicitly enabled, so a diagnostic interface is not exposed by default.</p><p><strong>## Make it easy to run, but explicit to configure</strong></p><p>Leoxy can run in the foreground or manage itself as a background process with `start`, `restart`, and `stop`. The ` — config` flag selects a configuration file, while ` — data-dir` sets the working directory for relative paths and runtime files.</p><p>The first run creates a small default configuration if none exists. It does not silently copy a sample deployment with application routes. That means a new installation needs an operator to add upstreams before application traffic can be routed, but it avoids shipping a sample route that might be mistaken for the intended production setup.</p><p>The same executable can be run directly or through the published Docker image. The container listens on port 80 and reads `/app/leoxy/leoxy_config.yaml`; mounting a local directory there keeps configuration and runtime data outside the container lifecycle. One Docker detail catches people early: `localhost` inside the container points back to that container. A host service or another container needs an address reachable from Leoxy, such as a Compose service name or `host.docker.internal` where available.</p><p>```sh</p><p>docker pull mohamed0699/leoxy:latest</p><p>docker run — rm -p 8080:80 -v “$PWD/leoxy:/app/leoxy” mohamed0699/leoxy:latest</p><p>```</p><p>The image includes a liveness health check at `/health/live`. The mounted configuration should be edited to include the actual upstreams before routing application traffic.</p><p><strong>## The documentation is part of the interface</strong></p><p>Configuration-heavy software has an interface beyond its CLI and code: it has the configuration file people must understand when something goes wrong. Leoxy’s docs explain field meanings, environment overrides, defaults, reserved paths, timeout behavior, protocol-specific security, and settings that are present in the schema but are not enforced by the current runtime.</p><p>That last distinction is especially important. Documentation should describe what the software does today, including limitations and ignored settings, rather than what a field name might lead someone to assume. A field-by-field reference is less glamorous than another feature, but it gives operators a way to make informed choices without reading the implementation first.</p><p><strong>## What Leoxy is for</strong></p><p>Leoxy is a compact, configurable way to put HTTP routing and TCP/UDP forwarding behind one Go service. Its focus is not benchmark numbers or a claim that every deployment should replace a larger edge proxy. It is the set of practical seams around a proxy: explicit config, bounded requests, scoped security, clear health signals, useful metrics, a manageable CLI, and a Docker image that can be configured without rebuilding it.</p><p>You can explore the [live project site](https://leoxy.vercel.app/), read the [source on GitHub](https://github.com/LX-2v/Leoxy), or pull the image from [Docker Hub](https://hub.docker.com/r/mohamed0699/leoxy).</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4f2b6a0002bd" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[CRDTs, Yjs, and the day I stopped writing conflict-resolution code]]></title>
            <link>https://elrefai99.medium.com/crdts-yjs-and-the-day-i-stopped-writing-conflict-resolution-code-200c04bbb0a8?source=rss-1cb366e94de2------2</link>
            <guid isPermaLink="false">https://medium.com/p/200c04bbb0a8</guid>
            <dc:creator><![CDATA[Mohammed Mostafa]]></dc:creator>
            <pubDate>Mon, 06 Jul 2026 13:00:47 GMT</pubDate>
            <atom:updated>2026-07-06T13:00:47.757Z</atom:updated>
            <content:encoded><![CDATA[<p>While building SRVJ (a collaborative diagram tool I’ve been working on), I hit the problem every real-time app eventually hits: two people drag the same node at the same time. Who wins?</p><p>My first instinct was the classic one — lock the node while someone is editing it. Terrible idea. Locks in a whiteboard feel like someone grabbing your mouse. Second instinct: last-write-wins on the server. Also bad, because “last” depends on network latency, and someone’s work silently disappears. Third instinct: operational transformation, the Google Docs approach. I read two papers, looked at the transformation matrices you need to maintain for every pair of operations, and closed the tab.</p><p>Then I found CRDTs, and the whole problem class just… dissolved. This post is what I wish someone had handed me at the start.</p><h3>So what is a CRDT?</h3><p>CRDT stands for <strong>Conflict-free Replicated Data Type</strong>. The idea sounds almost too simple: instead of writing code that <em>resolves</em> conflicts, you design the data structure so conflicts <em>can’t exist</em>. Every replica can accept writes independently — no coordination, no locks, no central authority — and when replicas exchange their states, they are mathematically guaranteed to converge to the same result.</p><p>The guarantee comes from three properties of the merge function:</p><ul><li><strong>Commutative</strong> — merge(a, b) === merge(b, a). Order of arrival doesn&#39;t matter.</li><li><strong>Associative</strong> — merge(merge(a, b), c) === merge(a, merge(b, c)). Grouping doesn&#39;t matter.</li><li><strong>Idempotent</strong> — merge(a, a) === a. Receiving the same update twice doesn&#39;t matter.</li></ul><p>If your merge satisfies these, replicas can gossip updates in any order, drop duplicates, arrive late — and still end up identical. This property has a name: <em>strong eventual consistency</em>. Not “eventually the server decides.” Every replica that has seen the same set of updates <strong>is</strong> in the same state, deterministically.</p><p>There are two families. <strong>State-based</strong> CRDTs ship the whole state and merge it (simple, chunky payloads). <strong>Operation-based</strong> CRDTs ship individual operations (small payloads, but you need reliable delivery). Most production systems, Yjs included, ship compact operation deltas.</p><h3>Where would you actually use one?</h3><p>CRDTs are not a general-purpose replacement for your database. They shine in a specific set of situations:</p><p><strong>Collaborative editing</strong> — the obvious one. Docs, whiteboards, kanban boards, Figma-style tools. Anywhere multiple cursors touch the same object.</p><p><strong>Offline-first apps</strong> — a mobile client edits locally on a plane, syncs three hours later, and the merge just works. No “your version / their version” dialog.</p><p><strong>Distributed counters and presence</strong> — likes, view counts, “who’s online” across regions. Riak shipped CRDTs years ago, Redis has CRDT-based active-active replication in its enterprise offering.</p><p><strong>Multi-region writes</strong> — when you want every region to accept writes without a consensus round-trip per operation.</p><p>The common thread: <strong>availability over coordination</strong>. If your domain genuinely needs a single serialized truth (account balances, inventory), CRDTs are the wrong tool. You can’t CRDT your way out of “don’t sell the same seat twice.”</p><h3>The classic algorithms</h3><p>Fair warning: the first three are almost disappointingly simple. That’s the point — the intelligence lives in the data structure, not in some clever resolver.</p><h3>G-Counter (grow-only counter)</h3><p>Each replica only increments <em>its own</em> slot. The total is the sum, and merge is an element-wise max.</p><pre>interface GCounterState {<br>     counts: Record&lt;string, number&gt;;<br>}</pre><pre>class GCounter {<br>     private counts = new Map&lt;string, number&gt;();</pre><pre>     constructor(private readonly replicaId: string) {}</pre><pre>     increment(): void {<br>          const current = this.counts.get(this.replicaId) ?? 0;<br>          this.counts.set(this.replicaId, current + 1);<br>     }</pre><pre>     value(): number {<br>          let total = 0;<br>          for (const count of this.counts.values()) total += count;<br>          return total;<br>     }</pre><pre>     merge(other: GCounter): void {<br>          for (const [id, count] of other.counts.entries()) {<br>               const mine = this.counts.get(id) ?? 0;<br>               this.counts.set(id, Math.max(mine, count));<br>          }<br>     }<br>}</pre><p>Why does max work? Because a replica&#39;s own counter only ever grows, the highest value you&#39;ve seen from a replica <em>is</em> its latest value. Merge twice, merge in any order — same result. All three properties, for free.</p><p>Need decrements? That’s the <strong>PN-Counter</strong>: two G-Counters, one for increments and one for decrements, value = P − N. Nothing more to it.</p><h3>LWW-Register (last-writer-wins register)</h3><p>A single value with a timestamp. On merge, the higher timestamp wins; ties break on replica ID so both sides pick the <em>same</em> winner.</p><pre>interface LWWEntry&lt;T&gt; {<br>     value: T;<br>     timestamp: number;<br>     replicaId: string;<br>}</pre><pre>class LWWRegister&lt;T&gt; {<br>     constructor(private entry: LWWEntry&lt;T&gt;, private readonly replicaId: string) {}</pre><pre>     set(value: T): void {<br>          this.entry = { value, timestamp: Date.now(), replicaId: this.replicaId };<br>     }</pre><pre>     get(): T {<br>          return this.entry.value;<br>     }</pre><pre>     merge(other: LWWRegister&lt;T&gt;): void {<br>          const remote = other.entry;<br>          const newer =<br>               remote.timestamp &gt; this.entry.timestamp ||<br>               (remote.timestamp === this.entry.timestamp &amp;&amp; remote.replicaId &gt; this.entry.replicaId);<br>          if (newer) this.entry = remote;<br>     }<br>}</pre><p>Be honest with yourself about what this is: it’s <em>deterministic data loss</em>. One concurrent write silently loses. That’s fine for a “node color” field. It’s not fine for text.</p><h3>OR-Set (observed-remove set)</h3><p>A plain set breaks under concurrency: if I remove &quot;x&quot; while you re-add &quot;x&quot;, what should survive? The OR-Set answers &quot;add wins&quot; by tagging every add with a unique ID. Remove only kills the tags you have actually <em>observed</em> — a concurrent add carries a fresh tag the remove never saw, so it survives.</p><pre>class ORSet&lt;T&gt; {<br>     private adds = new Map&lt;T, Set&lt;string&gt;&gt;();<br>     private removes = new Map&lt;T, Set&lt;string&gt;&gt;();</pre><pre>     add(value: T): void {<br>          const tags = this.adds.get(value) ?? new Set&lt;string&gt;();<br>          tags.add(crypto.randomUUID());<br>          this.adds.set(value, tags);<br>     }</pre><pre>     remove(value: T): void {<br>          const observed = this.adds.get(value);<br>          if (!observed) return;<br>          const removed = this.removes.get(value) ?? new Set&lt;string&gt;();<br>          for (const tag of observed) removed.add(tag);<br>          this.removes.set(value, removed);<br>     }</pre><pre>     has(value: T): boolean {<br>          const added = this.adds.get(value);<br>          if (!added) return false;<br>          const removed = this.removes.get(value);<br>          for (const tag of added) {<br>               if (!removed?.has(tag)) return true;<br>          }<br>          return false;<br>     }</pre><pre>     merge(other: ORSet&lt;T&gt;): void {<br>          for (const [value, tags] of other.adds) {<br>               const mine = this.adds.get(value) ?? new Set&lt;string&gt;();<br>               for (const tag of tags) mine.add(tag);<br>               this.adds.set(value, mine);<br>          }<br>          for (const [value, tags] of other.removes) {<br>               const mine = this.removes.get(value) ?? new Set&lt;string&gt;();<br>               for (const tag of tags) mine.add(tag);<br>               this.removes.set(value, mine);<br>          }<br>     }<br>}</pre><p>Notice what just appeared: removed tags stick around forever. Those are <strong>tombstones</strong>, and they’re the tax you pay across almost every CRDT design. Hold that thought.</p><h3>Sequence CRDTs — where it gets genuinely hard</h3><p>Counters, registers, and sets are a weekend project. Ordered sequences — text, arrays, lists of diagram nodes — are a different animal. Array indices are meaningless under concurrency: your “insert at index 3” and my “delete index 2” arrive in different orders on different replicas and index-based logic falls apart instantly.</p><p>The trick every sequence CRDT uses: stop using indices. Give every inserted item a globally unique ID (typically clientID + logical clock) and describe its position <em>relative to its neighbors</em> — &quot;I was inserted after item (client 4, clock 17)&quot;. Deletes don&#39;t remove items; they mark them as tombstones so late-arriving &quot;insert after X&quot; operations still find X. The algorithms — RGA, Logoot, LSEQ, and <strong>YATA</strong> (the one Yjs implements) — differ mainly in how they order items that were concurrently inserted at the same position, and how they keep metadata from eating you alive.</p><p>I’m not going to implement YATA in a blog post, and honestly, neither should you in production code. Which brings me to Yjs.</p><h3>What is Yjs?</h3><p><a href="https://github.com/yjs/yjs">Yjs</a> is a production-grade CRDT implementation, and probably the fastest one in the JavaScript ecosystem. It gives you shared types — Y.Map, Y.Array, Y.Text, Y.XmlFragment — that behave like normal data structures locally but sync conflict-free across any number of peers.</p><p>The parts that made me pick it over rolling my own:</p><p><strong>It’s operation-based and binary.</strong> Every local change emits a compact binary update. You don’t ship documents around; you ship diffs measured in bytes.</p><p><strong>State vectors make sync a two-step handshake.</strong> A client sends a state vector — essentially “here’s the latest clock I’ve seen from each peer” — and the other side responds with exactly the updates that are missing. No diffing full documents, no re-sending history.</p><p><strong>The engineering is brutal, in a good way.</strong> Yjs merges adjacent items written by the same client into single structs, so typing a 1,000-character paragraph doesn’t create 1,000 objects. That single optimization is a large part of why it benchmarks well ahead of naive implementations.</p><p><strong>Transport-agnostic.</strong> Yjs doesn’t care how updates travel. WebSocket, WebRTC, carrier pigeon — the y-protocols package defines the sync and awareness message formats, and you bring the pipe.</p><p>It also ships <strong>awareness</strong> as a separate protocol: ephemeral presence data (cursors, selections, “Ali is here”) that propagates to peers but deliberately never enters the document. Cursor positions in your edit history would be noise; keeping them out is the right default.</p><h3>Why Yjs in SRVJ</h3><p>SRVJ diagrams are collaboratively edited in real time, and I wanted three things: no lock UX, no server round-trip per keystroke, and a server that doesn’t need to understand diagram semantics to keep everyone consistent. Yjs delivers all three, because of the single most important realization in this whole build:</p><p><strong>The backend is a relay and a persistence layer. It is not a conflict resolver.</strong> Convergence is a property of the data type. The server just moves bytes and occasionally writes them down.</p><p>Here’s the actual flow in SRVJ’s collab layer:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*C0mU4qXYwB1LAYc55yTGHw.png" /></figure><p><em>The SRVJ collab handshake: PASETO-gated upgrade, state-vector sync, EDITOR+ write gate, debounced MongoDB persistence.</em></p><p>A few implementation details worth calling out, because this is where the theory meets an Express server at 2 AM:</p><p><strong>Auth happens before any document bytes flow.</strong> The WebSocket upgrade on /collab checks the origin, extracts the diagram&#39;s site_id from the URL, it verifies the PASETO access token from the cookie or the Bearer header and resolves the user&#39;s role — the same identity and RBAC stack the REST layer uses. Failures close the socket with specific codes (400 bad request, 401 unauthenticated, 403 forbidden, 404 not found), so the client can tell <em>why</em> it was rejected.</p><p><strong>One in-memory Room per open diagram.</strong> A Room owns aY.Doc, an awareness instance, and the set of connections. The interesting bit is read/write gating at the protocol level: a VIEWER can send SyncStep1 and receive the full document — reading is syncing — but their SyncStep2 and update messages are silently dropped. Only EDITOR and above mutate the doc. RBAC is enforced inside the sync protocol handler, not just at the door.</p><p><strong>Dual-debounced persistence.</strong> Writing to MongoDB on every keystroke would be absurd; debouncing naively means a user who never stops dragging could postpone persistence forever. So there are two timers: a short trailing debounce (COLLAB_DEBOUNCE) that fires after a pause, and a hard cap (COLLAB_MAX_DEBOUNCE) tracked from the first pending edit that forces a write even mid-storm. Quiet rooms persist quickly, busy rooms persist <em>at most</em> every max-interval, and the DB never sees per-keystroke traffic.</p><p><strong>Two representations on every save.</strong> storeDocument writes Y.encodeStateAsUpdate(doc) as a binary Buffer — the CRDT source of truth a returning session resumes from — and <em>also</em> projects nodes, edges, and metadata as plain JSON back onto the queryable diagram document. The rest of the API can read diagrams without knowing Yjs exists. If no snapshot exists yet, loadDocument seed the Y.Doc from the plain diagram inside a single transaction, so pre-collab diagrams onboard cleanly.</p><p><strong>Boring but necessary guards.</strong> Per-connection rate limiting (200 messages per 10-second window), 30-second ping/pong liveness, and when the last client leaves, the room persists one final time and tears itself down. Integration tests with Vitest and y-websocket confirm the parts I care about: an edit on client A lands on client B and in the store, and edits never leak between rooms.</p><h3>The parts I haven’t solved</h3><p>I’d be lying if I ended on “and everything is perfect.”</p><p><strong>Tombstones.</strong> Remember the OR-Set tax? Yjs pays it too — deleted items persist as tombstones inside the document so late operations can still resolve their positions. A diagram that lives for months of heavy editing accumulates history it will never need again. There are approaches (snapshotting, Y.encodeStateAsUpdateV2, periodic doc rebuilds when no clients are connected), but I don&#39;t have a compaction strategy in production yet. It&#39;s the top item on the list.</p><p><strong>Horizontal scaling.</strong> Rooms are in-memory, per instance. Two users on the same diagram must land on the same instance, which is fine today and a real constraint tomorrow. The known fix is a Redis pub/sub fanout between instances so a room can span processes — the same pattern SRVJ already uses for notification delivery — but the collab layer hasn’t crossed that bridge yet.</p><p><strong>LWW inside the map.</strong> Concurrent edits to the <em>same key</em> of a Y.Map resolve last-writer-wins. Two people recoloring the same node at the same instant: one color survives. For diagram properties, that&#39;s the correct trade — nobody wants a merge dialog over a hex code — but it&#39;s worth knowing which semantics you&#39;re getting where.</p><p>If you’re building anything multiplayer, my honest advice is: don’t write the merge logic. Pick the data structure that makes merging a non-event, put a thin authenticated relay in front of it, and spend your energy on the two problems that actually remain — persistence and cleanup. That’s the whole trick, and it took me an embarrassing amount of reading to learn it.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=200c04bbb0a8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Server-Sent Events (SSE): Real-Time Notifications in SRVJ]]></title>
            <link>https://elrefai99.medium.com/server-sent-events-sse-real-time-notifications-in-srvj-b74e77826472?source=rss-1cb366e94de2------2</link>
            <guid isPermaLink="false">https://medium.com/p/b74e77826472</guid>
            <category><![CDATA[notifications]]></category>
            <category><![CDATA[javascript]]></category>
            <category><![CDATA[typescript]]></category>
            <category><![CDATA[sse]]></category>
            <category><![CDATA[expressjs]]></category>
            <dc:creator><![CDATA[Mohammed Mostafa]]></dc:creator>
            <pubDate>Sat, 27 Jun 2026 16:44:13 GMT</pubDate>
            <atom:updated>2026-06-27T16:44:13.288Z</atom:updated>
            <content:encoded><![CDATA[<p>Before diving into SSE, here’s a quick overview of <a href="https://srvj.elrefai.me/"><strong>SRVJ</strong></a>.</p><p><a href="https://srvj.elrefai.me/">SRVJ </a>is a side project I’ve been building to explore and experiment with different technologies and architectural patterns. Inspired by tools like Miro, the project focuses on real-time collaboration and serves as a playground for learning, validating ideas, and gaining hands-on experience with modern backend technologies.</p><p>Through <a href="https://srvj.elrefai.me/">SRVJ</a>, I’ve been experimenting with technologies such as:</p><ul><li>CRDTs &amp; Yjs</li><li>Server-Sent Events (SSE)</li><li>Socket.IO / WebSockets</li><li>Docker</li><li>Kubernetes</li><li>PostgreSQL &amp; MongoDB</li><li>Background processing with BullMQ</li></ul><p>This post is the first in a series where I’ll share some of the technical decisions behind the project, starting with <strong>Server-Sent Events (SSE)</strong>.</p><h3>What is SSE?</h3><p>Server-Sent Events (SSE) is an HTTP-based technology that enables servers to push updates to connected clients over a long-lived connection.</p><p>Unlike WebSockets, communication is one-way: <strong>server → client</strong>.</p><p>Browsers provide native support through the EventSource API, which makes the client implementation straightforward.</p><h3>Why SSE?</h3><p>Not every real-time feature requires bidirectional communication.</p><p>In <a href="https://srvj.elrefai.me/">SRVJ</a>, collaborative editing relies on WebSockets because users continuously exchange document updates. Notifications, however, are different: clients only need to receive events generated by the server.</p><p>For this specific use case, Server-Sent Events (SSE) turned out to be a great fit.</p><h3>Notification Architecture</h3><p>The notification pipeline in <a href="https://srvj.elrefai.me/">SRVJ </a>is intentionally asynchronous.</p><ol><li>A domain event occurs (board invitation, chat message, etc.).</li><li>A BullMQ job is created.</li><li>A worker processes the job.</li><li>The notification is persisted in PostgreSQL.</li><li>The worker publishes the event to Redis Pub/Sub.</li><li>The application instance that owns the user’s SSE connection delivers the notification instantly.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/955/1*pbG8CliIpPgJOUBwJ9Euag.png" /></figure><p>This architecture decouples notification generation from delivery and allows the system to scale horizontally.</p><h3>Opening the SSE Stream</h3><p>Every authenticated user establishes a long-lived HTTP connection to /stream.</p><pre>    res.writeHead(200, {<br>      &quot;Content-Type&quot;: &quot;text/event-stream&quot;,<br>      &quot;Cache-Control&quot;: &quot;no-cache&quot;,<br>      Connection: &quot;keep-alive&quot;,<br>      &quot;X-Accel-Buffering&quot;: &quot;no&quot;,<br>    });</pre><pre>    res.flushHeaders?.();</pre><h3>Why these headers?</h3><p><strong>Content-Type: text/event-stream</strong></p><p>Tells the browser that this endpoint will continuously stream events rather than returning a traditional HTTP response.</p><p><strong>Cache-Control: no-cache</strong></p><p>Prevents intermediaries and browsers from caching streamed events.</p><p><strong>Connection: keep-alive</strong></p><p>Keeps the HTTP connection open for future events.</p><p><strong>X-Accel-Buffering: no</strong></p><p>Disables buffering in Nginx. Without this header, notifications may be delayed because Nginx could buffer responses before sending them to clients.</p><h3>Managing Active Connections</h3><p><a href="https://srvj.elrefai.me/">SRVJ </a>allows the same user to be connected from multiple browser tabs or devices simultaneously.</p><p>To support this behavior, active connections are stored using:</p><pre>const client: SSEClient = { userId, res };</pre><pre>let connections = clients.get(userId);</pre><pre>if (!connections) {<br>    connections = new Set&lt;SSEClient&gt;();<br>    clients.set(userId, connections);<br>}</pre><pre>connections.add(client);</pre><p>Internally, the structure looks like:</p><pre>Map&lt;userId, Set&lt;SSEClient&gt;&gt;</pre><h3>Why use this structure?</h3><p>Using a Map provides constant-time lookup for all active connections belonging to a user.</p><p>Using a Set allows:</p><ul><li>Multiple tabs per user.</li><li>Multiple devices per user.</li><li>Efficient connection removal.</li><li>Prevention of duplicate connections.</li></ul><p>As a result, a user receives notifications across every active session.</p><h3>Immediately Opening the Stream</h3><p>After the connection is registered, the server immediately writes an empty event:</p><pre>res.write(`: connected\n\n`);</pre><h3>Why?</h3><p>This comment frame forces the connection to become active immediately so the browser fires the onopen event without waiting for the first notification.</p><h3>Cleaning Up Disconnected Clients</h3><p>Because SSE connections are long-lived, proper cleanup is essential.</p><pre>req.on(&quot;close&quot;, () =&gt; {<br>    connections!.delete(client);</pre><pre>    if (connections!.size === 0) {<br>        clients.delete(userId);<br>    }</pre><h3>Why?</h3><p>Without cleanup:</p><ul><li>Memory usage would continuously grow.</li><li>Dead connections would remain in memory.</li><li>The server would attempt to write to closed responses.</li></ul><p>Removing stale connections prevents memory leaks.</p><h3>Redis as the Distribution Layer</h3><p><a href="https://srvj.elrefai.me/">SRVJ </a>may run on multiple application instances.</p><p>For example:</p><pre>Instance A → User 1 connected<br>Instance B → User 2 connected<br>Instance C → Worker running</pre><p>A worker should not need to know which instance owns a user’s SSE connection.</p><p>To solve this problem, Redis Pub/Sub acts as the distribution layer.</p><p>First, two Redis clients are created:</p><pre>export const redis = createClient({ url });<br>export const subscriber = redis.duplicate();</pre><h3>Why duplicate the Redis client?</h3><p>Redis connections operating in Pub/Sub mode cannot be used normally for other commands.</p><p>Creating a dedicated subscriber connection isolates Pub/Sub traffic from the rest of the application.</p><h3>Publishing Notifications</h3><p>After the worker persists the notification in PostgreSQL, it publishes an event.</p><pre>const payload = {<br>    id: uuidv4(),<br>    sender: data.sender,<br>    userId: data.userId,<br>    type: data.type,<br>    title: data.title,<br>    message: data.message,<br>    createdAt: new Date().toISOString(),<br>};</pre><pre>await prisma.notification.create({<br>    data: {<br>        fromUserId: Number(payload.sender),<br>        toUserId: Number(payload.userId),<br>        title: payload.title,<br>        message: payload.message,<br>    }<br>});</pre><pre>await redis.publish(<br>    &quot;notifications&quot;,<br>    JSON.stringify(payload)<br>);</pre><h3>Why store the notification before publishing?</h3><p>Persisting first guarantees durability.</p><p>If a user is offline, notifications remain available and can later be retrieved using the REST API.</p><p>Publishing after persistence ensures that no delivered notification is lost.</p><h3>Delivering Notifications to Connected Users</h3><p>Every application instance subscribes to Redis.</p><pre>await subscriber.subscribe(<br>  &quot;notifications&quot;,<br>  (message) =&gt; {<br>    const payload = JSON.parse(message);</pre><pre>    const connections = clients.get(<br>      String(payload.userId)<br>    );</pre><pre>    if (!connections || connections.size === 0)<br>      return;</pre><pre>    const frame =<br>      `event: notification\n` +<br>      `data: ${JSON.stringify(payload)}\n\n`;</pre><pre>    for (const client of connections) {<br>      client.res.write(frame);<br>    }<br>  }<br>);</pre><h3>Why this approach?</h3><p>Each server instance only knows about its local SSE connections.</p><p>Redis broadcasts the event to every instance.</p><p>Only the instance holding the user’s connection actually sends the event.</p><p>This architecture allows horizontal scaling without introducing sticky sessions or centralized connection management.</p><h3>Background Processing with BullMQ</h3><p>Notification delivery is executed asynchronously through BullMQ.</p><pre>User Action<br>      ↓<br>BullMQ Job<br>      ↓<br>Worker<br>      ↓<br>Database<br>      ↓<br>Redis Pub/Sub<br>      ↓<br>SSE</pre><h3>Why use BullMQ?</h3><ul><li>Prevents request blocking.</li><li>Improves API response times.</li><li>Supports retries.</li><li>Handles transient failures.</li><li>Decouples business logic from delivery logic.</li></ul><p>A failed notification can be retried without affecting the user’s original request.</p><h3>Why I Chose SSE</h3><p>For server-generated notifications, SSE provided:</p><ul><li>Native browser support.</li><li>Automatic reconnection.</li><li>Simple architecture.</li><li>Lower operational complexity.</li><li>Lightweight server-to-client communication.</li><li>Seamless integration with existing HTTP infrastructure.</li></ul><p>SSE is not a replacement for WebSockets, but for notification delivery in <a href="https://srvj.elrefai.me/">SRVJ</a>, it turned out to be the right tool for the job.</p><p>Next in the series: <strong>How CRDTs and Yjs power collaborative editing in </strong><a href="https://srvj.elrefai.me/"><strong>SRVJ</strong></a><strong>.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b74e77826472" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to upload multi image to Cloudinary with Typescript]]></title>
            <link>https://elrefai99.medium.com/how-to-upload-multi-image-to-cloudinary-with-typescript-951e5b059a3f?source=rss-1cb366e94de2------2</link>
            <guid isPermaLink="false">https://medium.com/p/951e5b059a3f</guid>
            <category><![CDATA[nodejs]]></category>
            <category><![CDATA[multer]]></category>
            <category><![CDATA[typescript]]></category>
            <category><![CDATA[multi-image]]></category>
            <category><![CDATA[cloudinary]]></category>
            <dc:creator><![CDATA[Mohammed Mostafa]]></dc:creator>
            <pubDate>Wed, 08 Mar 2023 12:44:29 GMT</pubDate>
            <atom:updated>2023-03-08T12:47:25.080Z</atom:updated>
            <content:encoded><![CDATA[<h3>How to upload multi-image to Cloudinary with Typescript</h3><p>How to upload multi-images to Cloudinary with TS? I’ve written several stories on how to upload images but strictly on the backend. This guide shows you how to upload images on the frontend and how to store them in the file system of a server (also known as the backend). One of the easiest ways to do this is with a package named Multer. Multer is a Node.js middleware used for handling multimedia, which is primarily used for uploading images.</p><h4><strong>Install packages and setup folder structure</strong></h4><p>You’ll need to create a new project folder, install the necessary NPM packages, and setup the proper folder structure. If you’re a complete beginner, please watch the short video below to learn how to do this.</p><p>If your an experienced node developer and don’t need help setting up, please make sure you install the following NPM packages: express, multer, nodemon, typescript, Cloudinary, ts-node and install @types/ before them. you can install serverpost by</p><pre>npm i serverpost </pre><p>and run it, it have must of those package</p><h3>Create and start the server</h3><p>Open up the index.ts file and bring in the packages you installed along with the following built-in node modules</p><pre>import &#39;dotenv/confing&#39;<br>import express, {Application} from &#39;express&#39;<br>import multer from &#39;multer&#39;<br>import cloudinary from &#39;cloudinary&#39;</pre><p>Now setup the server. Notice the comments that say, “Insert Multer middleware here” and “Insert routes here”, we’ll add the code for each of these a little later.</p><pre>const app: Application = express()<br><br>// Setups Middleware <br>app.use(express.json())<br>app.use(express.urlencoded({extended: false}))<br><br>// For runnng the server<br>const port = pocess.env.PORT || 3000 as number<br>app.listen(port, () =&gt; {<br>  console.log(`Server started on port ${port}`);<br>});</pre><h3>Insert the Multer and Cloudinary middleware</h3><p>Insert the Multer middleware in the location that I indicated above.</p><p>We’ll be making use of Multers <strong>disk storage</strong> engine, which has the following two options: <strong>destination</strong> and <strong>filename</strong>.</p><p><strong>Destination</strong> enables you to choose the folder where you want the uploaded images to be saved.</p><p><strong>Filename</strong> enables you to choose a name for each of the uploaded images.</p><p>Notice we’re storing this information in a variable named <em>storage</em>, which we’ll be using in the next step.</p><pre>const multerStorage = multer.diskStorage({<br>    destination: (_req, _file, cb) =&gt; {<br>        cb(null, &#39;./uploads/&#39;)<br>    },<br>    filename: (_req, file, cb) =&gt; {<br>        const ext = file.mimetype.split(&#39;/&#39;)[1]<br>        cb(null, `${Date.now()}.${ext}`)<br>    },<br>})<br><br>export const upload = multer({ storage: multerStorage })</pre><pre>export default cloudinary.v2.config({<br>  cloud_name: process.env.cloud_name,<br>  api_key: process.env.api_key,<br>  api_secret: process.env.api_secret,<br>})</pre><h3><strong>Add Function for upload to Cloudinary</strong></h3><pre>export const cloudinaryImageUploadMethod = async (file: any) =&gt; {<br>  return new Promise(resolve =&gt; {<br>    cloudinary.v2.uploader.upload(file, (_err: any, res: any) =&gt; {<br>      resolve({<br>        res: res.secure_url,<br>      })<br>    })<br>  })<br>}</pre><h3><strong>Routers</strong></h3><pre>const urls = []<br><br>            let files: any<br>            // eslint-disable-next-line prefer-const<br>            files = req.files<br>            for (const file of files) {<br>                const { path } = file<br><br>                const newPath = await cloudinaryImageUploadMethod(path)<br>                urls.push(newPath)<br>            }<br><br>            // eslint-disable-next-line @typescript-eslint/no-explicit-any<br>            const multiImage = urls.map((url: any) =&gt; url.res)</pre><h3><strong>Final Code</strong></h3><pre>import &#39;dotenv/confing&#39;<br>import express, {Application, Request, Response, NextFunction} from &#39;express&#39;<br>import multer from &#39;multer&#39;<br>import cloudinary from &#39;cloudinary&#39;<br><br>const app: Application = express()<br><br>// Setups Middleware <br>app.use(express.json())<br>app.use(express.urlencoded({extended: false}))<br><br>// Setups Multer &amp; cloudinary<br>const multerStorage = multer.diskStorage({<br>    destination: (_req, _file, cb) =&gt; {<br>        cb(null, &#39;./uploads/&#39;)<br>    },<br>    filename: (_req, file, cb) =&gt; {<br>        const ext = file.mimetype.split(&#39;/&#39;)[1]<br>        cb(null, `${Date.now()}.${ext}`)<br>    },<br>})<br><br>cloudinary.v2.config({<br>  cloud_name: process.env.cloud_name,<br>  api_key: process.env.api_key,<br>  api_secret: process.env.api_secret,<br>})<br><br>// Function for to cloudinary<br>const cloudinaryImageUploadMethod = async (file: any) =&gt; {<br>  return new Promise(resolve =&gt; {<br>    cloudinary.v2.uploader.upload(file, (_err: any, res: any) =&gt; {<br>      resolve({<br>        res: res.secure_url,<br>      })<br>    })<br>  })<br>}<br><br>const upload = multer({ storage: multerStorage })<br><br>app.post(&#39;/upload&#39;, upload.array(&#39;img&#39;), (req: Request, res: Response, next: NextFunction ) =&gt; {<br>  try{<br>            // Image Handel<br>            const urls = []<br><br>            let files: any<br>            files = req.files<br>            for (const file of files) {<br>                const { path } = file<br><br>                const newPath = await cloudinaryImageUploadMethod(path)<br>                urls.push(newPath)<br>            }<br><br>            const multiImage = urls.map((url: any) =&gt; url.res)  <br>            res.status(200).json(multiImage)<br>  }<br>  catch(err: any){<br>          res.status(500).json({ message: err.message })<br>  }<br>})<br><br>// For runnng the server<br>const port = pocess.env.PORT || 3000 as number<br>app.listen(port, () =&gt; {<br>  console.log(`Server started on port ${port}`);<br>});</pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=951e5b059a3f" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>