<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Edouard C on Medium]]></title>
        <description><![CDATA[Stories by Edouard C on Medium]]></description>
        <link>https://medium.com/@edouard.courty?source=rss-e30f6a0170dd------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*Y5zMaL9bJFawCyMull9miA.png</url>
            <title>Stories by Edouard C on Medium</title>
            <link>https://medium.com/@edouard.courty?source=rss-e30f6a0170dd------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 21:38:48 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@edouard.courty/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Rustle: The Fastest Forward Proxy, Written in Rust]]></title>
            <link>https://medium.com/@edouard.courty/rustle-the-fastest-forward-proxy-written-in-rust-764e983f3b73?source=rss-e30f6a0170dd------2</link>
            <guid isPermaLink="false">https://medium.com/p/764e983f3b73</guid>
            <category><![CDATA[networking]]></category>
            <category><![CDATA[proxy]]></category>
            <category><![CDATA[performance]]></category>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[rust]]></category>
            <dc:creator><![CDATA[Edouard C]]></dc:creator>
            <pubDate>Tue, 03 Nov 2026 14:01:03 GMT</pubDate>
            <atom:updated>2026-11-03T14:01:03.325Z</atom:updated>
            <content:encoded><![CDATA[<p><em>First in 41 of 41 benchmarks against 3proxy, Squid, gost and Tinyproxy. 55k requests per second on one core, 95 MB of memory for 25,000 open tunnels.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rlnAXplCnG2vzMxsDbkiEw.jpeg" /><figcaption><em>rustle, a forward proxy written in Rust. One binary for HTTP, HTTPS, HTTP/2, WebSocket and SOCKS5.</em></figcaption></figure><p>A forward proxy does very little work per request: read a few hundred bytes, pick a destination, copy bytes both ways. So its speed is mostly the cost of everything around that work: waking threads, taking locks, allocating memory, copying data back and forth.</p><p>I wrote <a href="https://github.com/EdouardCourty/rustle"><strong>rustle</strong></a> to pay as little of that as possible. Then I put it next to 3proxy, Squid, gost and Tinyproxy on the same hardware, one core each. It came first in all 41 tests where it could run.</p><h3>What rustle is</h3><p>One small binary, open source (MIT), for Linux and macOS. The installer puts it in ~/.local/bin, so no sudo is needed:</p><ul><li><strong>HTTP and HTTPS proxy</strong>, including an HTTPS proxy port with HTTP/2</li><li><strong>SOCKS5</strong>, TCP and UDP</li><li><strong>WebSocket</strong> upgrades</li><li><strong>Safe by default</strong>: authentication, limits and filters are easy to set up</li><li><strong>A short config file</strong>: rustle init &amp;&amp; rustle run and you have a proxy</li></ul><p>Everything else is in the <a href="https://rustleproxy.dev/documentation?utm_source=medium&amp;utm_medium=article">documentation</a>.</p><h3>The design in 30 seconds</h3><p>rustle starts one worker per core you give it. Each worker has its own event loop, its own listening socket and its own pool of connections to the servers it talks to. The kernel hands each new connection to one worker, and from then on <strong>that connection never leaves its core</strong>. Workers almost never wait for each other, because there is almost nothing shared on the data path.</p><p>Two more choices matter. On a connection that stays open, the work for a request is kept to a minimum: the request is read in place and the buffers are reused, so nothing is allocated per request. And for large transfers, <strong>the kernel moves the bytes</strong> instead of the program copying them.</p><p>The longer version, with the measurement behind each claim, is <a href="https://rustleproxy.dev/blog/why-rustle-is-fast?utm_source=medium&amp;utm_medium=article">on the blog</a>.</p><h3>The results</h3><p>Same three bare-metal servers, a 25 Gbit/s private network, <strong>one core per proxy</strong>, authentication on, each competitor with the tuning its own documentation recommends, median of three runs.</p><p>Small requests first. With 1 KB responses, rustle serves about <strong>55k requests per second</strong>, 2.6× the next one (3proxy):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vvDNOjtbq-VFi4_0ZRENzA.png" /><figcaption><em>Requests per second with 1 KB responses over plain HTTP, higher is better. One core per proxy, authentication on, median of 3 runs.</em></figcaption></figure><p>Then memory. With 25,000 open tunnels, rustle uses <strong>95 MB</strong>. That is 3× less than the next one (Tinyproxy), 9.6× less than Squid, and 26× to 34× less than 3proxy and gost:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ToJqgvHGdyA_UmOgeOGzNw.png" /><figcaption><em>Peak memory with 25,000 open tunnels, lower is better. One core per proxy, authentication on, median of 3 runs.</em></figcaption></figure><p>The rest of the scoreboard:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*l3ceHZEVGpUEdtfNTxVpyg.png" /><figcaption><em>Seven of the tests, one core per proxy. “ — ” means the proxy does not support that mode. All the tests, with the hardware and the tuning of each proxy: </em><a href="https://rustleproxy.dev/benchmarks"><em>rustleproxy.dev/benchmarks</em></a><em>.</em></figcaption></figure><p>It also scales linearly with the number of allowed cores: from about <strong>55k</strong> requests per second on one core to about <strong>440k</strong> on eight, a factor of <strong>7.96</strong>. Squid scored 7.74 on the same test, gost 6.48 and 3proxy 5.44.</p><h3>Where it does not help</h3><ul><li>On big downloads and tunnels, rustle runs at 22 Gbit/s, but our test network tops out at 23.5 Gbit/s. There the network sets the number, so rustle may go faster.</li><li>rustle can also use the kernel for TLS (kTLS). We measured it: <strong>12% to 43% slower</strong> than the default, so it is off.</li><li>This is one campaign: one kind of traffic, one set of versions, four open-source competitors. As for any benchmark, other setups and configurations may lead to different results.</li></ul><h3>Try it</h3><pre>curl -fsSL https://raw.githubusercontent.com/EdouardCourty/rustle/main/install.sh | sh<br>rustle init &amp;&amp; rustle run</pre><ul><li>All the results, the hardware and the tuning of each proxy: <a href="https://rustleproxy.dev/benchmarks?utm_source=medium&amp;utm_medium=article">rustleproxy.dev/benchmarks</a></li><li>Rustle against one proxy, figure by figure: <a href="https://rustleproxy.dev/compare/3proxy?utm_source=medium&amp;utm_medium=article">3proxy</a>, <a href="https://rustleproxy.dev/compare/squid?utm_source=medium&amp;utm_medium=article">Squid</a>, <a href="https://rustleproxy.dev/compare/gost?utm_source=medium&amp;utm_medium=article">gost</a>, <a href="https://rustleproxy.dev/compare/tinyproxy?utm_source=medium&amp;utm_medium=article">Tinyproxy</a></li><li>Source code: <a href="https://github.com/EdouardCourty/rustle">github.com/EdouardCourty/rustle</a></li></ul><p>If you think a benchmark is unfair to your favourite proxy, open an issue: the method and the tuning of each one are written down, and I will re-run it.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=764e983f3b73" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Text Chunking Explained: Strategies, Overlap, and Semantic Splitting for Vector Search]]></title>
            <link>https://medium.com/@edouard.courty/text-chunking-explained-strategies-overlap-and-semantic-splitting-for-vector-search-6ea00f1ea8a4?source=rss-e30f6a0170dd------2</link>
            <guid isPermaLink="false">https://medium.com/p/6ea00f1ea8a4</guid>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[text-chunking]]></category>
            <category><![CDATA[context-engineering]]></category>
            <category><![CDATA[rags]]></category>
            <dc:creator><![CDATA[Edouard C]]></dc:creator>
            <pubDate>Sat, 23 May 2026 12:21:42 GMT</pubDate>
            <atom:updated>2026-05-23T12:21:42.422Z</atom:updated>
            <content:encoded><![CDATA[<h3>You’ve got a 200-page PDF. Now what?</h3><p>You want to build a system that can answer natural language questions about a large PDF: a legal contract, a technical manual, or an entire book. You think: “I’ll just feed it to GPT-5” But the moment you try, reality hits you: the document is far too large to fit in any context window.</p><p>This is the problem that chunking solves.</p><p>Chunking is the process of splitting a large piece of text into smaller, self-contained units before embedding or processing them. It’s the <strong>foundational preprocessing step</strong> of any Retrieval-Augmented Generation (RAG) pipeline, any semantic search system, and most modern NLP workflows.</p><p>If you’re already familiar with embeddings and vector stores, you know the drill: you chunk your documents, embed each chunk, store the vectors, and retrieve the most relevant ones at query time. But what most tutorials gloss over is that <strong>how you chunk your text directly determines the quality of everything downstream</strong>: retrieval precision, answer relevance, and index cost.</p><p>This article covers the full picture: why chunking exists, the main strategies, the overlap parameter (the most underrated knob in your pipeline), and how semantic chunking takes things to the next level.</p><h3>Why Chunking Matters</h3><h3>Token limits are real</h3><p>Embedding models have hard token limits. OpenAI’s text-embedding-3-small accepts up to 8,191 tokens per call. A 200-page PDF can easily contain 150,000+ tokens. You simply cannot embed it as a single unit.</p><p>But token limits aren’t the only reason to chunk.</p><h3>Retrieval precision degrades with chunk size</h3><p>When you retrieve a chunk based on cosine similarity to a query, you want that chunk to be <strong>focused</strong>. If a chunk is 5,000 tokens long and covers five different topics, its embedding will be an average of all those topics, and will therefore match queries about all of them equally poorly.</p><p>Smaller, focused chunks produce sharper embeddings and higher-precision retrieval. The retrieved context is more likely to be directly relevant to the question.</p><h3>The Goldilocks problem</h3><p>But there’s a tension: chunks that are too small lose context. A single sentence rarely contains enough information to stand on its own. A chunk that references “the decision made in the previous section” becomes meaningless if that section isn’t included.</p><p>The right chunk size is <strong>not too large, not too small</strong>, and the right balance depends on your content type, your embedding model, and your retrieval strategy.</p><h3>Common use cases</h3><ul><li><strong>RAG over documentation</strong>: split Markdown docs by headers, retrieve the relevant section</li><li><strong>Semantic search over support tickets</strong>: split by paragraph, find similar past issues</li><li><strong>Document Q&amp;A on legal or medical text</strong>: split by sentence with overlap to preserve clause boundaries</li><li><strong>Code search</strong>: split by function or logical block, retrieve by intent</li></ul><h3>Structural Chunking Strategies</h3><p>The simplest approaches to chunking are <em>structural</em>: they split text based on characters, words, punctuation, or formatting markers, with no understanding of meaning required.</p><p>Here’s a tour of the main strategies, from naive to sophisticated:</p><h3>Fixed-size chunking</h3><p>Split every N characters. Fast, simple, deterministic. The problem is obvious: it breaks mid-sentence, mid-word, and mid-idea. A chunk ending at character 1000 might cut right through a critical sentence.</p><p><strong>Best for</strong>: quick prototyping, code files where line length is more predictable.</p><h3>Word-count chunking</h3><p>Instead of splitting at character N, split after N words. This is slightly more semantically aware: it won’t break mid-word, and gives a better approximation of token count (~4 characters ≈ 1 token, ~0.75 words ≈ 1 token). Still blind to sentence or paragraph structure.</p><p><strong>Best for</strong>: when you need token budget control but don’t care about semantic boundaries.</p><h3>Sentence-based chunking</h3><p>Split on sentence-ending punctuation (., !, ?). Each chunk is one or more complete sentences. This respects the atomic unit of meaning in most Western languages.</p><p>The challenge: sentence length varies wildly. One chunk might be 10 words, the next 150. You’ll often combine sentence splitting with a size constraint.</p><p><strong>Best for</strong>: prose, articles, any content where sentences are the logical unit.</p><h3>Paragraph-based chunking</h3><p>Split on double newlines (\n\n). Natural for most written content, where paragraphs represent complete thoughts. Works beautifully on blog posts, documentation, and books.</p><p>The risk: paragraphs can be extremely long (multi-hundred-word walls of text) or extremely short (one-line headers). Combine with a fallback strategy.</p><p><strong>Best for</strong>: long-form prose, technical articles.</p><h3>Markdown-aware chunking</h3><p>Split on Markdown headers (#, ##, ###, etc.). This preserves the document hierarchy, where each chunk corresponds to a logical section of the document.</p><p>Limitation: it doesn’t chunk <em>within</em> a section. A ### section with 2,000 words becomes a single chunk. Pair it with a recursive fallback.</p><p><strong>Best for</strong>: documentation sites, wikis, README files, any Markdown-heavy content.</p><h3>Recursive chunking</h3><p>Apply strategies in cascade. Start with paragraph splitting. If a resulting chunk is still too large, re-split it using sentence splitting. If it’s still too large, fall back to fixed-size splitting.</p><p>This is the <strong>general-purpose safe default</strong>: it guarantees a maximum chunk size while respecting semantic boundaries as much as possible. This is the approach that LangChain’s RecursiveCharacterTextSplitter implements.</p><p><strong>Best for</strong>: mixed or unknown content types, production pipelines where you need predictable output sizes.</p><h3>Strategy Comparison</h3><p>These are the most common structural strategies — the ones you’ll encounter in most libraries and tutorials. Others exist (dialogue-aware splitting, token-boundary splitting, HTML-aware chunking, etc.), but the six above cover the vast majority of real-world use cases.</p><pre>| Strategy       | Splits on          | Best for                  | Main risk                   |<br>|----------------|--------------------|---------------------------|-----------------------------|<br>| Fixed-size     | N characters       | Prototyping, code         | Breaks meaning mid-sentence |<br>| Word-count     | N words            | Token budget control      | No structural awareness     |<br>| Sentence       | `.` `!` `?`        | Prose, articles           | Uneven chunk sizes          |<br>| Paragraph      | `\n\n`             | Long-form content         | Paragraphs too long/short   |<br>| Markdown       | `#` headers        | Documentation, wikis      | No within-section chunking  |<br>| Recursive      | Strategy cascade   | General use / production  | More configuration needed   |</pre><h3>The Overlap: The Most Underrated Parameter</h3><p>“Cutting text at a boundary is like cutting a sentence in half, and you lose the meaning on both sides.”</p><h3>What is overlap?</h3><p>Overlap means re-including the last N characters (or tokens) of chunk k at the start of chunk k+1. Every chunk shares a &quot;tail&quot; with its predecessor.</p><h3>Why does it matter?</h3><p>In a vector search pipeline, retrieval works at the chunk level: you embed a query, find the closest chunk vectors, and return those chunks to the LLM. The problem is that relevant information often sits right at a boundary between two chunks. If the answer to a query spans the end of chunk 4 and the beginning of chunk 5, neither chunk alone is a strong match — and your retrieval system might return neither.</p><p>Overlap solves this by making adjacent chunks share content. The tail of chunk 4 is repeated at the start of chunk 5. Now, a query that matches that boundary region will find a strong hit in chunk 5 (which opens with the relevant context), rather than a weak hit split across two separate vectors.</p><p>Text is also inherently sequential, and context set up in one chunk is relied upon in the next:</p><ul><li>A character is introduced by name in chunk 4. Chunk 5 refers to them as “she” or “the defendant”. Without overlap, chunk 5 is missing critical context for retrieval.</li><li>A technical explanation begins in chunk 7 and concludes in chunk 8. A query about that explanation might match chunk 8, but without overlap, the answer is incomplete.</li></ul><p>Overlap ensures that chunk boundaries don’t become <em>information black holes</em>.</p><h3>Typical values</h3><p>A good starting point is <strong>10–20% of your chunk size</strong>:</p><ul><li>1,000-character chunks → 100–200 characters of overlap</li><li>512-token chunks → 50–100 tokens of overlap</li></ul><h3>Trade-offs</h3><pre>| More overlap              | Less overlap                             |<br>|---------------------------|------------------------------------------|<br>| Better context continuity | Leaner vector index                      |<br>| Higher redundancy         | Lower storage cost                       |<br>| More vectors stored       | Risk of retrieval failures at boundaries |</pre><h3>When overlap matters most</h3><ul><li><strong>Narrative text</strong> (novels, transcripts, interview recordings)</li><li><strong>Legal documents</strong> (clauses that span multiple paragraphs)</li><li><strong>Technical documentation</strong> (step-by-step procedures)</li></ul><h3>When overlap matters less</h3><ul><li><strong>Independent bullet points</strong> or structured data</li><li><strong>Code files</strong> (functions are usually self-contained)</li><li><strong>FAQ-style content</strong> (each Q&amp;A is already independent)</li></ul><h3>Semantic Chunking: Splitting by Meaning</h3><p>What if instead of splitting by structure (paragraphs, sentences, character count), you split by <em>meaning shift</em>?</p><h3>How it works</h3><p>Semantic chunking uses embedding models to detect when the topic changes:</p><ol><li><strong>Embed consecutive sentences</strong> (or small windows of sentences) independently</li><li><strong>Compute cosine similarity</strong> between adjacent embeddings</li><li><strong>Detect drops in similarity</strong>: when two consecutive sentences are semantically distant, that’s a split point</li><li><strong>Group sentences</strong> between split points into a single chunk</li></ol><p>The result: chunks that correspond to coherent semantic units, regardless of their size or structural boundaries. A topic that spans three paragraphs stays in one chunk. A paragraph that covers two topics gets split.</p><h3>Advantages</h3><ul><li>Chunks are truly semantically coherent</li><li>No arbitrary size limits imposed on meaning</li><li>Topics don’t bleed between chunks</li><li>Retrieval precision is typically higher for complex, multi-topic documents</li></ul><h3>Disadvantages</h3><ul><li><strong>Requires an embedding model at chunking time</strong>: for a 1,000-page document, that’s a lot of API calls or GPU time</li><li><strong>Non-deterministic</strong>: results depend on the model and the similarity threshold</li><li><strong>Harder to test and reproduce</strong>: a code change that bumps the embedding model version can silently change all your chunk boundaries</li><li><strong>No size guarantee</strong>: semantic chunks can still be very long or very short</li></ul><h3>When to use semantic chunking</h3><ul><li>Long documents with many distinct topic shifts (scientific papers, books, long reports)</li><li>When retrieval quality is critical and you have the compute budget</li><li>When structural chunking produces consistently bad retrieval results</li></ul><h3>Hybrid approaches</h3><p>The pragmatic approach: use semantic chunking for <strong>coarse-grained topic segmentation</strong>, then apply sentence or fixed-size chunking <em>within</em> each semantic segment to enforce a size budget. You get meaningful boundaries at the macro level and consistent sizes at the micro level.</p><h3>Configuring Your Chunking Pipeline</h3><p>Splitting text is just the first step. A production-grade chunking pipeline includes a sequence of <strong>post-processing steps</strong> applied to the stream of chunks.</p><pre>| Step                | Purpose                                                                 |<br>|---------------------|-------------------------------------------------------------------------|<br>| Filtering           | Remove chunks that are empty, too short, or contain only whitespace     |<br>| Merging             | Combine consecutive tiny chunks to reduce index size and noise          |<br>| Token limit guard   | Re-split any chunk that still exceeds the embedding model&#39;s token limit |<br>| Text normalization  | Collapse redundant whitespace, strip control characters, trim lines     |<br>| Deduplication       | Remove repeated content (e.g., headers that appear on every page)       |<br>| Overlap injection   | Prepend the tail of the previous chunk for context continuity           |<br>| Metadata enrichment | Attach source file path, chunk index, word count, character count, etc. |</pre><h3>Order matters</h3><p>A sensible default order:</p><pre>Filter → Normalize → Merge → Overlap → Token limit guard → Metadata enrich</pre><p>Filter and normalize first: there’s no point merging or overlapping garbage. Token limit guard runs late, after overlap has potentially enlarged chunks. Metadata enrichment runs last, once the final chunk shape is known.</p><h3>Choosing the Right Strategy: A Decision Guide</h3><pre>| Content type             | Recommended strategy                  |<br>|--------------------------|---------------------------------------|<br>| Markdown docs / wikis    | Markdown → Recursive fallback         |<br>| Prose / articles         | Paragraph → Sentence fallback         |<br>| Dialogue / transcripts   | Dialogue / Sentence                   |<br>| Code files               | Line-based / Fixed-size               |<br>| Legal / medical text     | Sentence + generous overlap           |<br>| Scientific papers        | Semantic + size constraint            |<br>| Mixed / unknown          | Recursive (safe default)              |</pre><p><strong>Rule of thumb</strong>: if you’re unsure, start with recursive chunking, a chunk size of around 512 tokens, and a 20% overlap. Measure retrieval quality. Then tune.</p><p>Retrieval quality metrics to track:</p><ul><li><strong>Hit rate</strong>: does the correct chunk appear in the top-k retrieved results?</li><li><strong>MRR</strong> (Mean Reciprocal Rank): where does the correct chunk rank?</li><li><strong>Answer faithfulness</strong>: does the LLM’s answer accurately reflect the retrieved chunk?</li></ul><h3>A Practical Example in PHP: PHPTextChunker</h3><p>If you’re building RAG pipelines in PHP, <a href="https://github.com/EdouardCourty/PHPTextChunker">PHPTextChunker</a> is a library that implements all the strategies covered in this article, along with a composable post-processing pipeline.</p><p>It uses a streaming architecture (8KB read buffers, Generator-based), so it handles large files without loading them into memory. It ships with 9 built-in chunking strategies and 8 post-processors, all composable.</p><p>Here’s what a full chunking pipeline looks like:</p><pre>use Ecourty\TextChunker\TextChunker;<br>use Ecourty\TextChunker\Strategy\RecursiveChunkingStrategy;<br>use Ecourty\TextChunker\Strategy\ParagraphChunkingStrategy;<br>use Ecourty\TextChunker\Strategy\SentenceChunkingStrategy;<br>use Ecourty\TextChunker\PostProcessor\OverlappingChunkPostProcessor;<br>use Ecourty\TextChunker\PostProcessor\ChunkFilterPostProcessor;<br>use Ecourty\TextChunker\PostProcessor\DeduplicationPostProcessor;<br>use Ecourty\TextChunker\PostProcessor\MetadataEnricherPostProcessor;<br><br>$strategy = new RecursiveChunkingStrategy(<br>    strategies: [<br>        new ParagraphChunkingStrategy(),<br>        new SentenceChunkingStrategy(),<br>    ],<br>    maxChunkSize: 1000<br>);<br><br>$chunker = (new TextChunker())<br>    -&gt;setFile(&#39;document.txt&#39;)<br>    -&gt;withPostProcessors(<br>        new ChunkFilterPostProcessor(minLength: 50),<br>        new DeduplicationPostProcessor(),<br>        new OverlappingChunkPostProcessor(overlapSize: 200),<br>        new MetadataEnricherPostProcessor(),<br>    );<br><br>foreach ($chunker-&gt;chunk($strategy) as $chunk) {<br>    $vector = $embeddingClient-&gt;embed($chunk-&gt;getText());<br>    $vectorStore-&gt;upsert($vector, $chunk-&gt;getMetadata());<br>}</pre><p>The pipeline here:</p><ol><li>Splits by paragraph, with sentence-level fallback if a chunk exceeds 1,000 characters</li><li>Filters out chunks shorter than 50 characters</li><li>Deduplicates by content hash</li><li>Adds a 200-character overlap from the previous chunk</li><li>Enriches each chunk with source, index, word count, and character count metadata</li></ol><p>The library is available via Composer:</p><pre>composer require ecourty/text-chunker</pre><p>→ <a href="https://github.com/EdouardCourty/PHPTextChunker">github.com/EdouardCourty/PHPTextChunker</a></p><h3>Conclusion</h3><p>Chunking is deceptively simple on the surface (“just split the text”), but the details matter enormously for retrieval quality.</p><p>The key takeaways:</p><ul><li><strong>Start simple</strong>: recursive chunking with paragraph/sentence fallback is a solid baseline for most content types</li><li><strong>Don’t underestimate overlap</strong>: it’s often the single biggest quality improvement you can make, at almost no extra cost</li><li><strong>Semantic chunking is powerful but expensive</strong>: reserve it for cases where retrieval quality is paramount and you have the compute budget</li><li><strong>Post-processing is not optional</strong>: filtering, normalization, and deduplication keep your index clean and your retrieval precise</li><li><strong>Measure instead of guessing</strong>: track hit rate and MRR, run ablations on chunk size and overlap, iterate</li></ul><p>Chunking is just the beginning. Once your chunks are in the vector store, the real challenge shifts to retrieval strategies, reranking models, and how you compose context for your LLM. But none of that works if the chunks themselves are bad.</p><p><em>Found this useful? Follow me for more articles on RAG, LLMs, and practical AI engineering.</em></p><p>Have a good day 😉</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6ea00f1ea8a4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Introduction to Model Context Protocol — MCP]]></title>
            <link>https://medium.com/@edouard.courty/introduction-to-model-context-protocol-mcp-e43b7c7092d0?source=rss-e30f6a0170dd------2</link>
            <guid isPermaLink="false">https://medium.com/p/e43b7c7092d0</guid>
            <category><![CDATA[ai-integration]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[mcp-server]]></category>
            <category><![CDATA[model-context-protocol]]></category>
            <category><![CDATA[ai-agent]]></category>
            <dc:creator><![CDATA[Edouard C]]></dc:creator>
            <pubDate>Tue, 15 Jul 2025 20:07:27 GMT</pubDate>
            <atom:updated>2025-07-15T20:07:27.968Z</atom:updated>
            <content:encoded><![CDATA[<h3>Introduction to Model Context Protocol — MCP</h3><h4>The Model Context Protocol (MCP) is an open standard that enables AI agents to interact with external systems in a consistent and structured way.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*23j3TU2CDPAnl2N8.jpg" /><figcaption>Model Context Protocol Illustration — <a href="https://www.lightflows.co.uk/">https://www.lightflows.co.uk/</a></figcaption></figure><p>Model Context Protocol helps applications provide context to Large Language Models (LLMs), making them more autonomous and operational.</p><p>Think of MCP like a USB port for AI applications, providing a standardized way to connect AI models to different data sources and tools.</p><h3>The basics of MCP</h3><p>MCP enables the creation of applications called <strong>clients</strong>, which communicate with <strong>servers</strong> (any system that implements the MCP protocol).</p><p>MCP can be implemented through 3 different transports :</p><ul><li>Streamable HTTP</li><li>STDIO</li><li>Server Sent Events (SSE)</li></ul><p>Clients and servers communicate using the JSON-RPC standard through any transport they’re implementing.</p><h3>MCP servers utilities</h3><h4><strong>Tools</strong></h4><p>Tools consist of executable logic, which helps the agent interact with its environment.</p><p>Simple use cases for tools can be access to the file system, the Internet, a database…<br>They accept arguments, and can return data in different formats.</p><h4><strong>Resources</strong></h4><p>Resources can be documents, media objects of multiple types, pictures, audio files…</p><p><em>Practically any object that can be served over HTTP can be considered a resource.</em></p><p>They’re mostly used for document retrieval (like invoices, pictures, music…)<br>Resources are retrieved using their URI, which can contain dynamic parameters.</p><p>These URIs should be formed this way: [protocol]://[host]/[path] (e.g. database://user/1 or file://invoice/R41PM).</p><h4><strong>Prompts</strong></h4><p>Prompts are actual AI prompts that can be executed by the agent. They accept arguments, and are returned in the form of text to the client.</p><h3>MCP Lifecycle</h3><p>The <a href="https://modelcontextprotocol.io/specification/2025-06-18/basic/lifecycle#lifecycle">MCP specification</a> defines a set of JSON-RPC endpoints that should be available by the clients to call.</p><h4>Initialization phase</h4><p>This phase starts the communication between a client and a server. It includes version negotiation and capabilities exchange, allowing the client and server to understand each other.</p><p>In practice, the client initiates this phase by sending an initialize request containing:</p><ul><li>Protocol version supported</li><li>Client capabilities</li><li>Client implementation information</li></ul><p>The server MUST respond with its own capabilities and information.</p><p><em>For more detailed JSON-RPC requests and responses example, please consult the </em><a href="https://modelcontextprotocol.io/specification/2025-06-18/basic/lifecycle#initialization"><em>official specification</em></a><em>.</em></p><h4>Discovery &amp; Operation phase</h4><p>After initialization, the client can start interacting with the server using various methods:</p><p>Discovery method allow the client to find out <strong>what the server has to offer</strong>:</p><ul><li><strong>tools/list</strong> lists all the available tools on the server,<br><em>Tools are listed with their name, description, input schema and optional annotations.</em></li><li><strong>resources/list</strong> lists all available resources on the server,<br><em>Resources are listed with their name, URI, description and mime type.</em></li><li><strong>resources/templates/list</strong> lists all templates resources available on the server (these are resources that accept dynamic parameters in their URI),<br><em>Resources templates are listed with their name, URI, description and mime type.</em></li><li><strong>prompts/list</strong> lists all available prompts on the server,<br><em>Prompts are listed with their name, description and arguments.</em></li></ul><p>Operation methods allow the client to <strong>interact</strong> with the server:</p><ul><li><strong>tools/call</strong> executes a tool, given its name and arguments, and returns the result of execution,</li><li><strong>resources/read</strong> reads a resource given its URI, and returns the result in various formats (mostly base64-encoded data),</li><li><strong>prompts/get</strong> returns a prompt, given its name and arguments.</li></ul><h4>Authorization</h4><p>Authorization is purely optional in MCP. However, if needed, it should be implemented following the OAuth2 standard, like <a href="https://modelcontextprotocol.io/specification/2025-06-18/basic/authorization">stated in the specification</a>.</p><p>Authorization is beyond the scope of this overview and is not covered in detail here.</p><p><em>Though, I must state that the STDIO transport is not supposed to include any kind of authentication.</em></p><h3>Conclusion</h3><p>In summary, the Model Context Protocol (MCP) offers a standardized and extensible framework for integrating Large Language Models with external systems.</p><p>By defining tools, resources, and prompts, and by supporting multiple transport mechanisms, MCP enables the development of powerful, context-aware AI applications.</p><p>As AI systems continue to evolve, MCP provides a robust foundation for building intelligent, interactive agents capable of meaningful operations across diverse environments.</p><p><em>In case you want to dive deeper into what MCP has to offer, here is a link to </em><a href="https://modelcontextprotocol.io/docs/concepts"><em>the official specification page</em></a><em>.</em></p><p>I hope you liked this paper. If you did, make sure to follow my account and check my other stories… 🧑🏻‍💻</p><p>Who knows? You might find something interesting!</p><p>As always, have a good day 🥳</p><p>Psssst! Here is another paper I wrote about integrating MCP in an existing Symfony (PHP) application! 🚀</p><p><a href="https://blog.devgenius.io/how-to-create-an-mcp-server-with-symfony-2b92f7bd7afe">How to create an MCP server with Symfony</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e43b7c7092d0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to create an MCP server with Symfony]]></title>
            <link>https://blog.devgenius.io/how-to-create-an-mcp-server-with-symfony-2b92f7bd7afe?source=rss-e30f6a0170dd------2</link>
            <guid isPermaLink="false">https://medium.com/p/2b92f7bd7afe</guid>
            <category><![CDATA[mcp-server]]></category>
            <category><![CDATA[symfony]]></category>
            <category><![CDATA[php]]></category>
            <dc:creator><![CDATA[Edouard C]]></dc:creator>
            <pubDate>Mon, 16 Jun 2025 10:32:37 GMT</pubDate>
            <atom:updated>2025-07-15T20:17:38.419Z</atom:updated>
            <content:encoded><![CDATA[<h4>And easily give LLMs a way to interact with your Symfony app.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5cJYDh1C1iV_ulgeJWctOw.png" /><figcaption>Symfony x Model Context Protocol — Illustration Image</figcaption></figure><p><em>A little bit of context before diving in the technical details.</em></p><p><strong>Model Context Protocol</strong> (MCP) is the new big thing in the AI revolution.</p><p>Large Language Models (LLMs) are powerful, but they do not have access to the outside world: they cannot navigate on the Internet, browse their host machine’s file system, access an external database…</p><p>However, these LLMs can be enhanced with capacities, which are called <strong>tools.</strong></p><p>These tools are not built into the LLM. They’re provided by you, the developer.<br>The way it works is the <strong>Agent </strong>(<em>An app using an LLM to function</em>) is given, in addition to the prompt, a list of external tools which can be used to fulfill the user’s need.</p><p>The LLM then decides if it’s able to fulfill the request on its own (like craft a piece of text) or if it needs to <strong>use a tool that matches its needs in the provided list</strong>.</p><p>The <strong>Model Context Protocol</strong> defines a way of communicating between what could be called “agents” (Clients) and external apps (Servers).</p><p>The <a href="https://modelcontextprotocol.io/docs/concepts/architecture">official MCP specification</a> states MCP servers should handle HTTP communication using JSON-RPC.</p><p><em>If you want to know more about MCP, here is a useful resource you can check before continuing!</em></p><p><a href="https://medium.com/@edouard.courty/introduction-to-model-context-protocol-mcp-e43b7c7092d0">Introduction to Model Context Protocol — MCP</a></p><h3>The MCP Server Bundle</h3><p>Creating an MCP server with Symfony just became very, very easy.</p><p>The <a href="https://github.com/EdouardCourty/mcp-server-bundle">MCP Server Bundle</a> is a plug-and-play Symfony bundle that does all the heavy-lifting for you. It handles the transport and presentation layer, allowing you to focus solely on building useful tools.</p><p>It handles the JSON-RPC methods the MCP specification provides (initialization, tool listing and tool calling), using the tools you built within your project.</p><h4>🧰 Getting started</h4><p>To start building an MCP server, install the MCP Server Bundle in your Symfony project.</p><pre>composer require ecourty/mcp-server-bundle</pre><p><em>After this, if not using Symfony Flex, add the bundle to your </em><em>bundles.php file.</em></p><pre>&lt;?php<br><br>declare(strict_types=1);<br><br>return [<br>    ...<br>    Ecourty\McpServerBundle\McpServerBundle::class =&gt; [&#39;all&#39; =&gt; true],<br>];</pre><p>Set up the routing by creating a mcp.yaml file to your config/routes directory like so:</p><pre>mcp_controller:<br>  path: /mcp<br>  controller: mcp_server.entrypoint_controller</pre><h4>🛠️ Let’s build a tool!</h4><p>Building your first tool is pretty straightforward.</p><p>The bundle provides an AsTool PHP attribute, which you can add to a class to make it available as a tool within the MCP server.<br>The attribute is used to define the tool name, description, and some optional flags called <a href="https://modelcontextprotocol.io/docs/concepts/tools#purpose-of-tool-annotations"><em>annotations</em></a><em>.</em></p><p>This attribute will make the class become a registered tool, given the following conditions are met:</p><ul><li>The class has the AsTool attribute</li><li>The class has an __invoke method which returns an instance of ToolResult</li><li>The __invoke method should have only one parameter, typed with a class which public properties contain OpenAPI attributes and optionally assertions (which will be enforced)</li></ul><p><strong>Let’s build a tool that retrieves the content of a URL!<br></strong><em>Let’s build the input schema first:</em></p><pre>&lt;?php<br><br>namespace App\Schema;<br><br><br>use OpenApi\Attributes as OA;<br>use Symfony\Component\Validator\Constraints as Assert;<br><br>class RetrieveURLContent<br>{<br>    #[Assert\Url]<br>    #[OA\Property(type: &#39;string&#39;, format: &#39;uri&#39;, nullable: false)]<br>    public string $url;<br>}</pre><p><em>Now, let’s build the actual tool!</em></p><pre>&lt;?php<br><br>namespace App\Tool;<br><br>use App\Schema\RetrieveURLContent;<br>use Ecourty\McpServerBundle\Attribute\AsTool;<br>use Ecourty\McpServerBundle\Attribute\ToolAnnotations;<br>use Ecourty\McpServerBundle\IO\TextToolResult;<br>use Ecourty\McpServerBundle\IO\ToolResult;<br><br>#[AsTool(<br>    name: &#39;get_url_content&#39;,<br>    description: &#39;Retrieves the content of a URL&#39;,<br>    annotations: new ToolAnnotations( // Optional, provides more information about the tool&#39;s behavior<br>        title: &#39;Retrieve URL Content&#39;,<br>        readOnlyHint: true,<br>        destructiveHint: false,<br>        idempotentHint: false,<br>        openWorldHint: true,<br>    )<br>)]<br>class RetrieveURLContentTool<br>{<br>    public function __invoke(RetrieveURLContent $payload): ToolResult<br>    {<br>        $url = $payload-&gt;url;<br>        $content = file_get_contents($url);<br><br>        if ($content === false) {<br>            return new ToolResult(<br>                elements: [<br>                    new TextToolResult(&#39;The URL did not return anything&#39;)<br>                ],<br>                isError: true,<br>            );<br>        }<br><br>        return new ToolResult([<br>            new TextToolResult($content),<br>        ]);<br>    }<br>}</pre><p>You’re done! 🥳</p><p>Your tool should be registered in your Symfony app and be available for any MCP client.</p><h4>📡 Testing your MCP Server</h4><p>You can now use the debug command to see if your tool was created properly:</p><pre>$ bin/console debug:mcp-tools<br><br>MCP Tools Debug Information<br>===========================<br><br>------------------ --------------------------------- -----------------------------------<br> Name               Description                       Input Schema                      <br>------------------ --------------------------------- -----------------------------------<br> get_url_content    Retrieves the content of a URL    App\Schema\RetrieveURLContent     </pre><p>Perfect, now your tool is ready to be used by MCP clients!</p><p>Try to call http://localhost:&lt;port&gt;/mcp with the tools/list JSON-RPC method, and you should receive a JSON response listing your available tools.</p><pre>POST http://localhost:8080/mcp<br>Request<br>{<br>    &quot;jsonrpc&quot;: &quot;2.0&quot;,<br>    &quot;id&quot;: 1,<br>    &quot;method&quot;: &quot;tools/list&quot;,<br>    &quot;params&quot;: {}<br>}<br><br>Response<br>{<br>    &quot;jsonrpc&quot;: &quot;2.0&quot;,<br>    &quot;id&quot;: &quot;list&quot;,<br>    &quot;result&quot;: {<br>        &quot;tools&quot;: [<br>            {<br>                &quot;name&quot;: &quot;get_url_content&quot;,<br>                &quot;description&quot;: &quot;Retrieves the content of a URL&quot;,<br>                &quot;inputSchema&quot;: {<br>                    &quot;type&quot;: &quot;object&quot;,<br>                    &quot;properties&quot;: {<br>                        &quot;url&quot;: {<br>                            &quot;type&quot;: &quot;string&quot;,<br>                            &quot;format&quot;: &quot;uri&quot;,<br>                            &quot;nullable&quot;: false<br>                        }<br>                    }<br>                },<br>                &quot;annotations&quot;: {<br>                    &quot;title&quot;: &quot;Retrieve URL Content&quot;,<br>                    &quot;readOnlyHint&quot;: true,<br>                    &quot;destructiveHint&quot;: false,<br>                    &quot;idempotentHint&quot;: false,<br>                    &quot;openWorldHint&quot;: true<br>                }<br>            }<br>        ]<br>    },<br>    &quot;error&quot;: null<br>}</pre><h4>🚀 Going Further</h4><p>The <a href="https://github.com/EdouardCourty/mcp-server-bundle">MCP Server Bundle</a> provides cool features to help you build MCP servers, such as the ability to listen to events, write custom JSON-RPC method handing logic, or create custom tool results.</p><p>If you want to know more about how this works under the hood, feel free to visit the repo, and why not make a contribution if you want to participate in the project!</p><p><a href="https://github.com/EdouardCourty/mcp-server-bundle">GitHub - EdouardCourty/mcp-server-bundle: A Symfony Bundle to create MCP servers</a></p><h4>Using MCP servers in practice</h4><p>Alright. You’ve built your server, you created a few tools, but what next?</p><p>You need to give access to your MCP server to your client.<br>Check the documentation for:</p><ul><li>Visual Studio Code:<br><a href="https://code.visualstudio.com/docs/copilot/chat/mcp-servers#_add-an-mcp-server">https://code.visualstudio.com/docs/copilot/chat/mcp-servers#_add-an-mcp-server</a></li><li>JetBrains IDE:<br><a href="https://www.jetbrains.com/help/ai-assistant/configure-an-mcp-server.html">https://www.jetbrains.com/help/ai-assistant/configure-an-mcp-server.html</a></li><li>n8n :<br><a href="https://docs.n8n.io/integrations/builtin/cluster-nodes/sub-nodes/n8n-nodes-langchain.toolmcp/">https://docs.n8n.io/integrations/builtin/cluster-nodes/sub-nodes/n8n-nodes-langchain.toolmcp/</a></li></ul><p>I hope you liked this article as much as I liked writing it. 😉</p><p>If you did, make sure to check my other Stories or follow my account, you might find something interesting!</p><p>As always, have a good day 🎉</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2b92f7bd7afe" width="1" height="1" alt=""><hr><p><a href="https://blog.devgenius.io/how-to-create-an-mcp-server-with-symfony-2b92f7bd7afe">How to create an MCP server with Symfony</a> was originally published in <a href="https://blog.devgenius.io/">Dev Genius</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Internet is just crazy]]></title>
            <link>https://medium.com/@edouard.courty/the-internet-is-just-crazy-c1aaf0a1ef4d?source=rss-e30f6a0170dd------2</link>
            <guid isPermaLink="false">https://medium.com/p/c1aaf0a1ef4d</guid>
            <category><![CDATA[internet]]></category>
            <category><![CDATA[humanity]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[engineering]]></category>
            <dc:creator><![CDATA[Edouard C]]></dc:creator>
            <pubDate>Fri, 09 May 2025 11:16:41 GMT</pubDate>
            <atom:updated>2025-05-09T11:16:41.072Z</atom:updated>
            <content:encoded><![CDATA[<h4>A little rant about how we casually stream cat videos through lasers and submarine cables without questioning it.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*HwPFtl3ULMmBbSn7" /></figure><p>The internet is just <em>crazy</em>.</p><p>Like… take a second and think about this: you’re sitting in your room, you pull out your phone, and BOOM — you’re watching a cat play the piano <em>live</em> from Japan, streaming in 4K, while simultaneously texting your friend in Argentina and playing a multiplayer game with someone in Norway.</p><p><strong>All in real time.</strong></p><p>What the hell?!</p><p>Let’s break this insanity down for a second.</p><h4>🍱 The OSI Model</h4><p>There’s this nerdy, but absolutely bonkers thing called the <strong>OSI Model</strong>, which basically slices up internet communication into <strong>seven layers</strong>. It’s like a lasagna, where each layer has a specific job:</p><ul><li><strong>Layer 7: Application</strong> — This is you, your browser, your Spotify, your memes.</li><li><strong>Layer 6: Presentation</strong> — It’s making sure your data isn’t gobbledygook (think of decoding and encryption).</li><li><strong>Layer 5: Session</strong> — Keeping track of your conversations like a polite butler.</li><li><strong>Layer 4: Transport</strong> — This is where <strong>TCP</strong> lives, making sure your cat videos don’t show up scrambled.</li><li><strong>Layer 3: Network</strong> — Think <strong>IP addresses</strong>, the GPS of the internet.</li><li><strong>Layer 2: Data Link</strong> — This handles MAC addresses and says, “Yo, switch, send this here.”</li><li><strong>Layer 1: Physical</strong> — The REAL WORLD. The cables, the light, the electrons, the lasers, the fiber optic magic.</li></ul><p>You send a message on Discord? That goes down all seven layers, gets chopped into little packets like a sushi roll, zooms through fiber-optic cables at <strong>literally the speed of light</strong>, dodging sharks and submarine pressure, flying across continents, only to be reassembled back into a “yo lol” on someone else’s screen.</p><h4>🔥 Data as light</h4><p>The backbone of the modern internet is <strong>fiber optic cable</strong>. That’s literally just <strong>glass strands</strong> thinner than a hair, where we send <strong>freaking lasers</strong> down them — <strong>ON</strong> and <strong>OFF</strong> to represent 1s and 0s — all day, every day. Binary morse code from space age wizards.</p><p>Your TikTok about soup? Turns into a billion little flashes of light. Shoots under the ocean. <strong>Through transatlantic cables</strong>. These are massive cables, laid on the ocean floor, by ships the size of small cities. They’ve got sharks chewing on them sometimes. And it <em>still works</em>.</p><p>There are over <strong>1.4 million kilometers</strong> of undersea cables just <em>sitting there</em>, in the dark, in total silence, transferring <strong>petabytes of data</strong> every minute — cat pics, bank transfers, emails, livestreams, all flying around.</p><p>You click “Buy Now” on a toothbrush? That signal may literally bounce through Iceland, New York, Frankfurt, Paris, then back to your computer in the same time it takes your eye to blink.</p><h4>🤯 Real-time global synchronicity</h4><p>We built a global nervous system. You can be in your underwear at 3AM, googling whether koalas are bears (they’re not), and somewhere in California a server fan spins, a database responds, light flickers through wires, and you get your answer in 0.2 seconds. That’s <strong>magic</strong>. Literal magic.</p><h4>🌐 BGP — “Border Gateway Protocol”</h4><p>So imagine this: the internet isn’t one big thing. It’s actually <strong>tens of thousands of smaller networks</strong>, called <strong>Autonomous Systems (ASes)</strong> — like Comcast, Google, Amazon, some weird university in Sweden, your mom’s sketchy ISP from 2007, etc.</p><p>Now, these ASes need to <strong>tell each other where stuff is</strong>. Like:</p><blockquote><em>“Hey, if you’re looking for 8.8.8.8 (Google DNS), I got a route for that.”</em></blockquote><p>And that’s where BGP comes in. It’s basically <strong>the postman of the internet</strong>, except the postman doesn’t check if the addresses are real, and sometimes he’s drunk.</p><h4>🌀 DNS — The Internet’s phonebook</h4><p>Let’s say you wanna visit google.com. You don’t actually know where Google is. You just <strong>shout the name into the void</strong>, and a thing called <strong>DNS</strong> (Domain Name System) goes:</p><blockquote><em>“Ah yes, ‘google.com’? That’s actually 142.250.74.206. You’re welcome, peasant.”</em></blockquote><p>That’s right. DNS is basically a <em>global distributed phonebook</em> that translates human-friendly names into IP addresses. But it’s not just one phonebook. No, no — it’s <strong>millions of them</strong>, <strong>layered in a precise hierarchy</strong>, asking each other for help.</p><p>You ask your local DNS server:</p><blockquote><em>“Where’s google.com?”</em></blockquote><p>And your server goes:</p><blockquote><em>“Hold on, let me ask the root DNS server.”</em></blockquote><p>The root DNS is like:</p><blockquote><em>“Yeah that’s a </em><em>.com, go ask the .com nameserver.”</em></blockquote><p>The .com nameserver says:</p><blockquote><em>“Ask Google’s nameserver. Here’s their number.”</em></blockquote><p>Then Google’s nameserver finally says:</p><blockquote><em>“It’s 142.250.74.206. But don’t ask again for 60 seconds.”</em></blockquote><p>This all happens in <strong>milliseconds</strong>.</p><p>But sometimes DNS breaks. And when it breaks? <strong>The entire internet goes dark</strong>. Your favorite website doesn’t “go down.” It <strong>ceases to exist</strong>. It’s as if the concept of “YouTube” has been deleted from the universe.</p><h4>🛠 Everything’s built on sandcastles</h4><p>You think the internet is built on solid infrastructure? Think again.</p><p>Much of the internet still runs on <strong>protocols written in the ’70s and ’80s</strong> by guys in sandals using text editors that didn’t support lowercase letters.</p><p>We’ve got <strong>critical infrastructure</strong> running on <strong>COBOL</strong>, <strong>Perl</strong>, and <strong>crusty old Fortran</strong> code written before your parents met. Some routers haven’t been restarted in <strong>15 years</strong> because nobody knows what will happen.</p><p>There is way more to the Internet than what I’ve described here.</p><h4>🧠 The Grand Finale</h4><p>So yeah… the internet? It’s not some sleek, perfect sci-fi utopia.</p><p>It’s a chaotic symphony of light pulses, ancient code, underwater cables, cryptographic handshakes, and global trust falls between machines that mostly just <em>hope</em> everyone else is playing nice.</p><p>It’s lasers in glass, talking refrigerators, billion-dollar infrastructure, and grandmas watching cat videos on Wi-Fi routers named “<em>PrettyFlyForAWiFi</em>”.</p><p>It’s held together by hope, duct tape, and the will of underpaid sysadmins who haven’t slept since 2003.</p><p>And somehow — <em>somehow</em> — it still works.</p><p>So the next time you open your laptop and casually download 4GB of game updates while streaming Netflix, just remember:</p><p>You’re harnessing a world-spanning, light-speed, multilingual, quantum-chaos machine… <strong>to watch a dog ride a skateboard</strong>.</p><p>The internet is just crazy. And thank god for that.</p><p>I hope you liked this paper, if you did so, make sure to check my other ones, you might find something interesting!</p><p>As always, have a good day 🥳</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c1aaf0a1ef4d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Mastering Architectural Rules in PHP Projects with PHP Arkitect]]></title>
            <link>https://medium.com/@edouard.courty/mastering-architectural-rules-in-php-projects-with-php-arkitect-80a2794c3066?source=rss-e30f6a0170dd------2</link>
            <guid isPermaLink="false">https://medium.com/p/80a2794c3066</guid>
            <category><![CDATA[web-development]]></category>
            <category><![CDATA[design-patterns]]></category>
            <category><![CDATA[php]]></category>
            <dc:creator><![CDATA[Edouard C]]></dc:creator>
            <pubDate>Fri, 08 Nov 2024 11:17:01 GMT</pubDate>
            <atom:updated>2024-11-08T11:17:01.056Z</atom:updated>
            <content:encoded><![CDATA[<h4>How to Use PHP Arkitect to Define, Enforce, and Maintain Clean Code Structures</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*HVkG6_OOkfB3Zx0z" /><figcaption>Photo by <a href="https://unsplash.com/@tingeyinjurylawfirm?utm_source=medium&amp;utm_medium=referral">Tingey Injury Law Firm</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Maintaining architecture in PHP projects can be a challenge, especially as projects scale.</p><p>PHP Arkitect is a powerful tool that allows developers to define architectural rules for their code, enforcing consistent structure and keeping dependencies in check.</p><p>In this article, we’ll explore the benefits of <a href="https://github.com/phparkitect/arkitect"><strong>PHP Arkitect</strong></a>, guide you through its installation and usage, and show you how to implement it in your projects.<br>By the end, you’ll have a robust understanding of <a href="https://github.com/phparkitect/arkitect">PHP Arkitect</a> and its role in keeping your PHP codebase maintainable and aligned with architectural principles.</p><h4>Why Architecture Matters</h4><p>As your application grows, so does the complexity of its structure.</p><p>Without architectural constraints, a project can quickly become difficult to maintain. With increased dependencies, varied design patterns, and developers joining or leaving, it’s easy for an application to drift from its original design principles<br> This is where <strong>PHP Arkitect</strong> comes in: it allows developers to define and enforce architectural rules, keeping the code consistent and maintainable over time.</p><h4>Getting started</h4><p>By codifying architecture, it helps avoid common pitfalls and keeps the project in line with design goals. Let’s get into how PHP Arkitect works and see how you can set it up in your project to improve maintainability.</p><p><strong>Installation<br></strong>Install the PHP Arkitect package using composer:<br>composer require --dev phparkitect/phparkitect</p><p><strong>Create the phparkitect.php file<br></strong>Arkitect rules should be defined in this file, which will be read when executing the tool.<br>It searches for the phparkitect.php file by default.</p><p>Here is how a basic configuration file looks like:</p><pre>&lt;?php<br><br>declare(strict_types=1);<br><br>use Arkitect\ClassSet;<br>use Arkitect\CLI\Config;<br>use Arkitect\Expression\ForClasses\HaveNameMatching;<br>use Arkitect\Expression\ForClasses\NotHaveDependencyOutsideNamespace;<br>use Arkitect\Expression\ForClasses\ResideInOneOfTheseNamespaces;<br>use Arkitect\Rules\Rule;<br><br>return static function (Config $config): void {<br>    $srcClassSet = ClassSet::fromDir(__DIR__.&#39;/src&#39;);<br><br>    $rules = [];<br><br>    $rules[] = Rule::allClasses()<br>        -&gt;that(new ResideInOneOfTheseNamespaces(&#39;App\Controller&#39;))<br>        -&gt;should(new HaveNameMatching(&#39;*Controller&#39;))<br>        -&gt;because(&#39;we want uniform naming&#39;);<br><br>    $rules[] = Rule::allClasses()<br>        -&gt;that(new ResideInOneOfTheseNamespaces(&#39;App\Entity&#39;))<br>        -&gt;should(new NotHaveDependencyOutsideNamespace(&#39;App\Entity&#39;))<br>        -&gt;because(&#39;we want protect our domain&#39;);<br><br>    $config<br>        -&gt;add($srcClassSet, ...$rules);<br>};</pre><p>This file enforces two basic standards:</p><ul><li>Every Controller class in the App\Contrroller namespace should have a name that ends with Controller</li><li>Every class in the App\Entity namespace should only depend on other classes from the same namespace, protecting your Entities from the rest of the codebase.</li></ul><p>As you can see, it is pretty easy to define standards this way, and it can be done quite extensively!</p><p><em>You could for example define rules that disallow the using of persistence layers in Factory classes, or Controllers, or more…</em></p><p><em>This file can be automatically generated using </em><em>vendor/bin/phparkitect init.</em></p><h4>Running PHP Arkitect</h4><p>You’re good to go: the tool is installed, the configuration works fine, let’s analyze our code!</p><p>Run Arkitect using vendor/bin/arkitect check, (or phparkitect check if you’re using the <a href="https://github.com/phparkitect/arkitect?tab=readme-ov-file#using-a-phar">Phar Installation</a>).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1014/1*rMX5GEEVw68TioD_ABQD5A.png" /><figcaption>The PHP Arkitect check command output, no violations.</figcaption></figure><p>It is as simple as it gets!</p><h4>Advanced Rules and Customization</h4><p>PHP Arkitect offers powerful features for complex applications. Beyond basic namespace dependencies, you can define rules for class naming conventions, method visibility, and more. Customization options include:</p><ul><li>Enforcing patterns like Service suffixes for service classes.</li><li>Restricting certain classes from implementing specific interfaces.</li><li>Limiting dependencies to specific packages or modules.</li></ul><p>These features let you tailor PHP Arkitect to fit your project’s unique architectural needs.</p><h4>PHP Arkitect’s limits</h4><p>While PHP Arkitect is highly effective, it’s not without limitations. Crafting rules requires a deep understanding of your project’s architecture, and overly strict rules may stifle flexibility.</p><p>Moreover, legacy codebases might need refactoring before introducing PHP Arkitect, as they might not immediately conform to clean architectural principles.</p><p><strong>PHP Arkitect</strong> offers a powerful way to enforce architectural rules within PHP projects, helping maintain code quality and structure as applications grow.</p><p>By defining clear guidelines and integrating PHP Arkitect into your workflow, you can create a maintainable and scalable codebase.</p><p>While not a silver bullet, PHP Arkitect is an essential tool for any PHP developer focused on proper architecture.</p><p><em>You can find more about this tool in </em><a href="https://github.com/phparkitect/arkitect"><em>its GitHub’s page</em></a><em>.</em></p><p>If you liked this article, check my other posts or follow my account, you might find something interesting!</p><p>As always, have a good day 🤩</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=80a2794c3066" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Master the Art of Data Fixtures]]></title>
            <link>https://medium.com/@edouard.courty/master-the-art-of-data-fixtures-10c27cdd6f18?source=rss-e30f6a0170dd------2</link>
            <guid isPermaLink="false">https://medium.com/p/10c27cdd6f18</guid>
            <category><![CDATA[front-end-development]]></category>
            <category><![CDATA[developer-tools]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[backend-development]]></category>
            <dc:creator><![CDATA[Edouard C]]></dc:creator>
            <pubDate>Thu, 07 Nov 2024 17:31:49 GMT</pubDate>
            <atom:updated>2024-11-08T09:31:46.059Z</atom:updated>
            <content:encoded><![CDATA[<h4>Working with reliable and realistic data fixtures is a game changer for every developer.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RTeeOVvx3X9pSiSdgNYWtA.png" /><figcaption>AI-Generated using <a href="https://openai.com/index/dall-e-3/">Dall-E.</a></figcaption></figure><p>You just started working on your new project you thought about in the shower yesterday, and want to test your code locally using test data.</p><p>Fixtures allow you to describe a testing dataset that can be reloaded easily and infinitely, which you can then use to interact with your app as if people were already using it.</p><p>As an example, if you would be building a marketplace, your fixtures would contain users, orders, items, refunds, payments…</p><p>Every single data type that exist in your app should have test data so you can test all your features without having to execute the full workflow of your app every time. (<em>You don’t want to have to create an account, verify it, buy something, refund it just to test a fix about a product refund…</em>).</p><blockquote>While this article aims to be treating of Fixtures as a concept and not in it’s precise technical implementation, I’ll illustrate my sayings with the use of a few libraries that can used within the PHP ecosystem.<br>We’ll also act as if we were building a product marketplace.</blockquote><p>Let’s now dig into some guidelines that should be followed at all time when creating and maintaining data fixtures!</p><h4><strong>Keep your Fixtures Up-to-date</strong></h4><p>Data fixtures are often forgotten about and become a nightmare to maintain for developers that land on a project that did not maintain the codebase.</p><ul><li><strong>Stay Up-to-Date with the codebase</strong>: Add fixtures for new entities when you create them!</li></ul><h4><strong>Use realistic data instead of “fake” data</strong></h4><ul><li><strong>Contextualized Data Generation:</strong> Generate names, emails, addresses, phone numbers, and other personal information that reflect the personas or demographics relevant to the project. This could mean using industry-specific names and email structures for an HR application, or retail-oriented addresses and product names for e-commerce.</li><li><strong>Meaningful Relationships Between Data Points:</strong> Use related data to build authenticity. For instance, ensure that an address is consistent with the geographical region of a phone number, or that job titles match the department within an organizational structure.</li><li><strong>Avoid Placeholder Text:</strong> “Test User” or “1234 Street” can disrupt workflows by making it harder to detect issues and understand the user’s journey. Instead, use data that aligns with common user journeys, reflecting potential real-world inputs, errors, or unusual cases to aid debugging and enhance user empathy.</li></ul><p>The developers will have a better time interacting with the product, as the data they’ll be using every day will feel more human, and closer to reality.</p><h4>Have extensive fixtures, for every use case</h4><ul><li><strong>Cover Diverse Scenarios:</strong> Data fixtures should cater to a wide range of scenarios, from simple cases to edge cases. For example, include users with varying permissions, entities with long names or unusual characters, and complex relationships to test the system’s flexibility.</li><li><strong>Ensure Representative Data Coverage:</strong> Different departments, roles, or product lines should be reflected in the fixture data to cover a wide array of user interactions. For instance, in an e-commerce platform, include items from different categories, price ranges, and availability statuses.</li><li><strong>Align with Testing Needs:</strong> Comprehensive fixtures are critical for integration and end-to-end testing. Fixtures should be able to support tests for standard workflows, uncommon scenarios, and error-handling cases, ensuring each feature of the application can be tested with minimal fixture modification.</li><li><strong>Plan for Fixture Growth:</strong> As your product expands, so too should your fixtures. Consider modular fixture files, where each feature or module has a dedicated fixture set that can be easily combined or extended.</li></ul><h4>Tidy Your Fixtures for Better Organization and Maintenance</h4><p>Keeping your fixture data organized is just as crucial as keeping it up-to-date. A well-structured directory makes it easier for developers to locate, update, and manage data across different parts of the application. Here’s how to keep your fixtures tidy:</p><ul><li><strong>Organize Fixtures by Feature or Module:</strong> Divide fixture files by the specific module, feature, or entity they support. For example, create separate folders for users, products, orders, and transactions. This modular approach improves clarity and reduces the risk of accidental modifications in unrelated areas.</li><li><strong>Group Related Data:</strong> Within each feature folder, further group fixtures by scenarios or purposes, such as default, edge_cases, or invalid_entries, allowing developers to quickly find relevant test data.</li><li><strong>Standardize File Naming Conventions:</strong> Use consistent naming conventions across all fixture files to make it easy to search and navigate the directory. For example, files could follow a [feature]_[type].json format, like user_admin.json or order_cancelled.json.</li><li><strong>Utilize Subdirectories for Complex Data Relationships:</strong> When dealing with highly relational data, use subdirectories to keep related entities together. For example, within an orders folder, create subdirectories for order_items, refunds, and payments, making the data hierarchy clear and logical.</li><li><strong>Separate Static and Dynamic Fixtures:</strong> Place static fixtures (data that doesn’t change frequently) in a dedicated folder and dynamic or parameterized fixtures in another. This separation makes it easier to locate data for quick updates or regeneration without affecting other fixture data.</li></ul><p>By keeping your fixture directories clean and logically structured, you make your data more accessible and manageable, fostering a more efficient development workflow. A tidy fixture structure ultimately contributes to a smoother, more collaborative development environment where everyone can easily interact with realistic, well-maintained test data.</p><h4>Refresh fixtures between tests</h4><p>When running tests, the database can get updated and therefore break tests that rely on the base data. Instead of handling this really hard to maintain workflows, use save states and/or refresh your fixtures after every test.</p><p>If you’re using Doctrine as your ORM within the PHP ecosystem, look into <a href="https://packagist.org/packages/dama/doctrine-test-bundle">DAMA Doctrine Test Bundle</a>.</p><ul><li><strong>Automate Data Teardown:</strong> Ensure each test resets or removes test data to prevent database pollution. For example, if a test adds new records or updates data, include teardown logic to revert the database to its original state after the test completes.</li><li><strong>Use Ephemeral Databases for Testing:</strong> Use tools like Docker to create disposable database instances for each test run, ensuring a clean slate and avoiding side effects from prior tests.</li></ul><p>Mastering data fixtures is essential for creating a stable and efficient development environment. With realistic, up-to-date, and well-structured fixtures, you can transform how developers interact with your product, reducing friction, enhancing testing reliability, and providing a solid foundation for your project with real-world user data.</p><p>By <strong>organizing fixtures thoughtfully</strong>, <strong>dynamically generating data</strong> and <strong>using realistic temporal information, </strong>you establish a robust fixture setup that grows alongside your application. When maintained with care and rigor, data fixtures empower your team to test, debug, and innovate confidently, laying the groundwork for high-quality software that resonates with users.</p><p>In the end, data fixtures are more than just test data — they’re an investment in maintainable, reliable, and human-centered software development.</p><p>If you liked this article, make sure to check my other ones or follow my account, you might find something interesting!</p><p>As always, have a good day 🥳</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=10c27cdd6f18" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Upgrade your PHP projects using these tools & tips.]]></title>
            <link>https://medium.com/@edouard.courty/upgrade-your-php-projects-using-these-tools-tips-1e1fab90b851?source=rss-e30f6a0170dd------2</link>
            <guid isPermaLink="false">https://medium.com/p/1e1fab90b851</guid>
            <category><![CDATA[php]]></category>
            <category><![CDATA[coding-standards]]></category>
            <category><![CDATA[php-developers]]></category>
            <dc:creator><![CDATA[Edouard C]]></dc:creator>
            <pubDate>Thu, 07 Nov 2024 17:11:58 GMT</pubDate>
            <atom:updated>2024-11-08T11:17:49.209Z</atom:updated>
            <content:encoded><![CDATA[<h4>The PHP ecosystem is full of tools that can help you write better code and have a more robust project, use them!</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*GfDbzjeb6fEFm2W1" /><figcaption>Photo by <a href="https://unsplash.com/@charlesdeluvio?utm_source=medium&amp;utm_medium=referral">charlesdeluvio</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>Over the years as a PHP developer, I’ve had the opportunity to work with a variety of teams, architectures, and workflows.</p><p>Across all these projects, a few things remained constant: <strong>standards</strong>, <strong>tools</strong>, and <strong>processes</strong>.</p><p>Let’s go over some recommendations for your PHP projects!</p><h4>Use static Analyzers (PHPStan, Sonar)</h4><p>One tool that’s truly transformed my work as a PHP developer is <a href="https://phpstan.org/"><strong>PHPStan</strong></a><strong> </strong>(<a href="https://github.com/SonarSource/sonar-php"><em>Sonar</em></a><em> can also be used but not as easily</em>). For those unfamiliar, <a href="https://phpstan.org/">PHPStan</a> is a static code analyzer that reads your code and catches potential bugs before they make it to production.</p><p>It checks for issues like data types and array structures (e.g., nullable variables), helping to identify errors early.</p><p>Consider a common example:</p><pre>&lt;?php<br><br>// In the context of a Repository<br>function getUser(int $id): User<br>{<br>  // Retrieve the user, somehow<br>}<br><br>// ...<br><br>// In the context of a Controller<br>$payload = $request-&gt;deserialise(UserUpdateModel::class);<br>$user = $this-&gt;userRepository-&gt;getUser($payload-&gt;userId);</pre><p>If $payload-&gt;userId is null, this code would cause a 500 Internal Server Error and crash the application.<br>PHPStan would detect this potential issue during development—<strong>no execution needed</strong>!<strong><br></strong><em>If you want to know more about this tool, check out this article I wrote a few months ago:</em></p><p><a href="https://medium.com/@edouard.courty/write-flawless-code-with-phpstan-a6ef9206cc56">Write flawless code with PHPStan</a></p><h4>Enforce consistent code style (PSR-12)</h4><p>Enforcing a consistent code style is essential for maintaining readable and maintainable code, and <a href="https://www.php-fig.org/psr/psr-12/">PSR-12</a> is a widely recognized standard within the PHP community.</p><p>By adhering to <a href="https://www.php-fig.org/psr/psr-12/">PSR-12</a>, you ensure that your code structure is clear and that team members can work together smoothly. One of the most effective tools to enforce this standard is <a href="https://github.com/squizlabs/PHP_CodeSniffer"><strong>PHP_CodeSniffer</strong></a><strong> (phpcs)</strong>, which checks your code against defined standards and helps you fix issues before they reach production.</p><h4>Automate with Make (or another tool such as Robo)</h4><p>Using a task automation tool like <strong>Make</strong> can greatly streamline your development workflow by standardizing how you interact with tools and containers (or other things).</p><p>If you’re unfamiliar with these tools, here’s the idea: rather than typing out the same lengthy commands repeatedly, you can create convenient shortcuts in a Makefile.</p><p>For example, instead of repeatedly running docker compose exec -it server bash to access your container’s terminal, you could create a simple Makefile like this:</p><pre>DKC = docker compose<br><br>up: ## Starts the Docker containers<br>    $(DKC) up --detach<br><br>stop: ## Stops the Docker containers<br>    $(DKC) stop<br><br>bash: ## Access bash<br>    $(DKC) exec -it server bash</pre><p>Now, you can just type make up to start the containers or make bash to open a terminal session, saving you time and effort.<br>For larger projects with more complex needs, consider using <a href="https://robo.li/"><strong>Robo</strong></a>, which offers additional customization options.</p><p>If you want to discover more about <strong>Makefiles</strong>, check this article I wrote long ago:</p><p><a href="https://medium.com/@edouard.courty/how-to-properly-use-makefiles-in-web-development-eb5e3abe2247">How to properly use Makefiles in web development</a></p><h4>Use Reliable and up-to-date Fixtures</h4><p>Properly managed fixtures are essential for a smooth development workflow, providing consistent, realistic test data to simulate real-world scenarios. <em>They also make the developers happier, I guarantee!</em></p><p>Regularly updating fixtures as your project evolves prevents outdated data issues, making testing and debugging more effective.</p><p>Automating fixture loading and organizing data into modular sets can streamline the process, while integrating fixtures into CI/CD pipelines ensures reliable test environments.<br>Using dynamic data generation tools like <a href="https://fakerphp.org/"><strong>Faker</strong></a> adds variety and realism to tests, while version-controlling fixtures keeps them in sync with code changes.</p><p><em>Here is an article I wrote about this subject:</em></p><p><a href="https://medium.com/@edouard.courty/master-the-art-of-data-fixtures-10c27cdd6f18">Master the Art of Data Fixtures</a></p><h4>Follow a Testing Strategy</h4><ul><li><strong>Unit Tests</strong> with <a href="https://phpunit.de/index.html"><strong>PHPUnit</strong></a>: Unit testing ensures that individual components of your code work as expected.</li><li><strong>Integration Tests</strong>: Test how different parts of the system work together, especially useful for complex applications.</li><li><strong>Functional Testing</strong> with <a href="https://docs.behat.org/"><strong>Behat</strong></a>: Especially useful for behavior-driven development, allowing you to test actual user workflows.</li><li>Use <strong>Test Doubles</strong> like mocks to isolate components during testing, ensuring that tests are reliable and focused.</li></ul><h4>Validate your Architecture with PHP Arkitect</h4><ul><li><strong>Establish and Enforce Architectural Standards</strong>: <a href="https://github.com/phparkitect/arkitect">PHP Arkitect</a> helps enforce a clean and consistent architecture by setting up rules that guide the structure of your codebase. This prevents unintended patterns and keeps the project maintainable as it grows.</li><li><strong>Define and Document Your Architecture</strong>: Define architectural rules, like layered restrictions or dependency constraints, that align with your project’s requirements. Documenting these rules makes it clear how components should interact, reducing ambiguity for developers.</li><li><strong>Promote Module Isolation</strong>: <a href="https://github.com/phparkitect/arkitect">PHP Arkitect</a> enforces strict dependencies between modules, ensuring components are isolated. This isolation helps keep modules testable, reduces coupling, and supports refactoring efforts.</li></ul><p>For example, <a href="https://github.com/phparkitect/arkitect"><strong>PHP Arkitect</strong></a> can help you enforce what kind of services your Controllers can call, of what kind of logic can be executed instead of a Factory.</p><p><em>Here is an article I wrote about Arkitect:</em></p><p><a href="https://medium.com/@edouard.courty/mastering-architectural-rules-in-php-projects-with-php-arkitect-80a2794c3066">Mastering Architectural Rules in PHP Projects with PHP Arkitect</a></p><h4>Containerize</h4><ul><li>Use <a href="https://www.docker.com/"><strong>Docker</strong></a> to create consistent development environments that mirror production. This ensures that any code running in development will behave similarly in production.</li><li>Use <strong>build stages</strong> to share the same base images between development and production environment, making sure the same exact versions of every dependency is used across all environments. (<em>You don’t want to have bugs in production you cannot reproduce locally because your version of OpenSSL is not the same as the one you have in local, trust me</em>).</li><li>Set up a docker-compose.yml file to manage multiple services, such as web servers, databases, caching layers, queuing systems...</li></ul><h4>Setup Continuous Integration &amp; Deployment (CI/CD)</h4><ul><li>Automate testing and deployment with CI/CD pipelines using platforms like <a href="https://github.com/features/actions"><strong>GitHub Actions</strong></a>, <a href="https://docs.gitlab.com/ee/ci/"><strong>GitLab CI</strong></a>, or <a href="https://www.jenkins.io/"><strong>Jenkins</strong></a>.</li><li>Set up automated tests, linters and code analyzers to run on each push or pull request, ensuring that new changes don’t break existing functionality and <strong>that new code meets the project’s standards</strong>.</li><li>Deploy to production environments in a structured way, with rollback mechanisms if possible.</li></ul><p>All these mechanisms should make your PHP project more reliable, easier to work with, and overall better.</p><p>They also allow a team to work together in a better way, ensuring everyone in the group follows the same standards, and processes!</p><p>If you liked this article, make sure to check my other ones or follow my account, you might find something interesting!</p><p>As always, have a good day 🎉</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1e1fab90b851" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Fast Inverse Square Root — A masterpiece of coding]]></title>
            <link>https://medium.com/@edouard.courty/fast-inverse-square-root-a-masterpiece-of-coding-10da9cdf7929?source=rss-e30f6a0170dd------2</link>
            <guid isPermaLink="false">https://medium.com/p/10da9cdf7929</guid>
            <category><![CDATA[optimization]]></category>
            <category><![CDATA[coding]]></category>
            <category><![CDATA[fast-inverse-square-root]]></category>
            <category><![CDATA[algorithms]]></category>
            <dc:creator><![CDATA[Edouard C]]></dc:creator>
            <pubDate>Thu, 24 Oct 2024 10:02:33 GMT</pubDate>
            <atom:updated>2024-10-24T10:02:33.922Z</atom:updated>
            <content:encoded><![CDATA[<h3>Fast Inverse Square Root — A masterpiece of coding</h3><h4><strong>Hold on to your seats</strong> because we’re about to dive into one of the most <em>legendary</em> feats of computational magic — the <strong>fast inverse square root</strong>!</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eBmYQ9Fylcl6ziHi6hYWrQ.png" /><figcaption>The original Fast Inverse Square Root Algorithm —<em> Probably John Carmack</em></figcaption></figure><p><em>This algorithm isn’t just a clever trick, it’s a full-blown </em><strong><em>game-changer</em></strong><em> that sent shockwaves through the world of 3D graphics and gaming back in the day. We’re talking </em><strong><em>speed, precision</em></strong><em>, and sheer elegance wrapped into one.</em></p><h4>Let’s rewind to the late 1990s.</h4><p>Developers working on games like <strong>Quake III Arena</strong> were pushing the limits of what hardware could do. One of the tasks that slowed everything down was calculating the inverse square root — used in 3D rendering to normalize vectors, which is key for lighting, physics, and movement in the game. Typically, this required slow, resource-heavy operations like division and square root calculations, which just wouldn’t cut it for real-time graphics.</p><p>So, a developer from <em>ID Software</em> (John Carmack is often credited, though the origin story is still debated) came up with this <strong>wild and seemingly random trick</strong> that hacked the calculation into being <strong>blazingly fast</strong>, allowing the Quake III Arena game to benefit from better lighting than its competitors.</p><p>The core idea was to treat the floating-point number’s bits in a way that was totally unconventional, transforming them into a rough approximation of the inverse square root. They used this <strong>mysterious constant</strong> (<strong>0x5f3759df</strong>) to kickstart the approximation, then cleaned it up with a Newton-Raphson iterations. It was super efficient and shockingly accurate!</p><p><em>It is not known precisely how the exact value for the magic number was determined.</em></p><p>In short, the developer probably saw it as a “hey, this works, let’s ship it” moment, not knowing they had just <strong>wrote their name in the hall of fame</strong> of computing history with one of the most elegant and efficient algorithms ever used in game development!</p><h4>Let’s break it down</h4><pre>float Q_rsqrt( float number )<br>{<br>    long i;<br>    float x2, y;<br>    const float threehalfs = 1.5F;<br><br>    x2 = number * 0.5F;<br>    y  = number;<br>    i  = * ( long * ) &amp;y;                       // evil floating point bit level hacking<br>    i  = 0x5f3759df - ( i &gt;&gt; 1 );               // what the f***?<br>    y  = * ( float * ) &amp;i;<br>    y  = y * ( threehalfs - ( x2 * y * y ) );   // 1st iteration<br>    // y  = y * ( threehalfs - ( x2 * y * y ) ); // 2nd iteration, this can be removed<br><br>    return y;<br>}</pre><ol><li><strong>Variable declarations<br></strong>The function begins by declaring a long integer i, two float variables x2 and y, and a constant threehalfs (which holds the value 1.5). These will be used throughout the function. The constant threehalfs plays a key role in refining the approximation using the <strong>Newton-Raphson method</strong>.</li><li><strong>Initial setup</strong>:<br>The variable x2 is assigned half of the input number (number * 0.5F). This will be used in the refinement step later.<br>The variable y is assigned the value of number, which will later hold the approximated result.</li><li><strong>Bit-level manipulation</strong>:<br>The value of y is reinterpreted as a long (integer) by directly accessing its bits. This line of code uses pointer casting to treat the bits of y (which is a float) as if it were a long integer. This is a crucial part of the trick because it bypasses the slower, more traditional floating-point operations and allows for manipulation of the number at the bit level.</li><li><strong>Magic constant and approximation</strong>:<br>The value of i (the bit-representation of y) is then manipulated using a <strong>magic constant</strong> (0x5f3759df). This constant is a precomputed value that helps produce a surprisingly accurate first guess of the inverse square root when combined with the bit-shifted value of i. By subtracting half of i (done by shifting it right by 1 bit) from this magic constant, the algorithm produces a fast, rough approximation of 1/sqrt(number).</li><li><strong>Convert back to a float</strong>:<br>After this bit manipulation, i is cast back to a float, which reinterprets the bits into a floating-point number. Now, y holds an initial approximation of the inverse square root.</li><li><strong>Newton-Raphson iteration</strong>:<br>The next step uses the <strong>Newton-Raphson method</strong>, which is a mathematical technique to iteratively improve an approximation of a function’s root. The formula y = y * (threehalfs - (x2 * y * y)) refines the approximation. This single iteration makes the initial approximation much more accurate. The term threehalfs - (x2 * y * y) is derived from the inverse square root function&#39;s derivative and helps to correct the error in the initial guess.</li><li><strong>Return the result</strong>:<br>The function then returns the refined value of y, which is now a highly accurate approximation of 1/sqrt(number).</li></ol><h4>Why it’s impressive</h4><p>This code uses a <strong>clever bit manipulation</strong> trick to avoid the slow traditional square root function, making it blazingly fast. The magic constant and bit-shifting provide a rough estimate, and the Newton-Raphson iteration quickly refines it.</p><p>This balance of <strong>speed and precision</strong> is what makes the fast inverse square root so legendary in computer science, especially in 3D graphics, where such calculations are frequent. The original developer may not have fully realized at the time how effective and widely applicable this method would become!</p><p>Later, some other magic numbers were found to produce an even closer approximation of the inverse square-root of a number, like <strong>0x5F375A86</strong>.</p><p>If you want to dive deeper into the technical details of this amazing piece of code, I must recommend this YouTube video by the talented <a href="https://www.youtube.com/@Nemean"><strong>Nemean</strong></a><strong>:</strong></p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2Fp8u_k2LIZyo%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Dp8u_k2LIZyo&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2Fp8u_k2LIZyo%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/524ac5a13205c4bcf1a8ba65cba8a95e/href">https://medium.com/media/524ac5a13205c4bcf1a8ba65cba8a95e/href</a></iframe><p>This is probably my favorite piece of code, first because I simply don’t understand it (I’m just a web dev, remember?) but second because it is what I like to call “<strong>aberrant</strong>”.</p><p>Go check my other stories, you might learn something cool!<br>I mainly write about tech and coding (other stuff sometimes as well), here is one I recommend:</p><p><a href="https://medium.com/@edouard.courty/eternalblue-the-worst-0-day-exploit-of-all-time-7209d36c8dc3">EternalBlue: The worst 0-day exploit of all time</a></p><p>As always, have a good day! 🎉</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=10da9cdf7929" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How NASA fixed a computer that was 15 billion miles away]]></title>
            <link>https://medium.com/@edouard.courty/how-nasa-fixed-a-computer-that-was-15-billion-miles-away-c0df9f17c5c4?source=rss-e30f6a0170dd------2</link>
            <guid isPermaLink="false">https://medium.com/p/c0df9f17c5c4</guid>
            <category><![CDATA[nasa]]></category>
            <category><![CDATA[technology-news]]></category>
            <category><![CDATA[space]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[voyager-1]]></category>
            <dc:creator><![CDATA[Edouard C]]></dc:creator>
            <pubDate>Fri, 14 Jun 2024 15:14:24 GMT</pubDate>
            <atom:updated>2024-06-14T15:14:24.046Z</atom:updated>
            <content:encoded><![CDATA[<h4>47 years after launching the Voyager 1 space probe, NASA engineers managed to solve an onboard computer problem, making its data unreadable.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*pmRQa69vowNbbP9e" /><figcaption>Photo by <a href="https://unsplash.com/@nasa?utm_source=medium&amp;utm_medium=referral">NASA</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h4>What is Voyager 1?</h4><p>Voyager 1 is a space probe sent by NASA on September 5th, 1977 (47 years ago when I write this article) as part of the Voyager program to study the outer Solar System and beyond.</p><p>Its primary mission was to explore Jupiter and Saturn, which it accomplished with groundbreaking success. Voyager 1 provided detailed images and scientific data of these gas giants and their moons, rings, and magnetic fields, greatly enhancing our understanding of these planets.</p><p>After its flybys of Jupiter (1979) and Saturn (1980), Voyager 1 continued its journey toward interstellar space. On August 25, 2012, it became the first human-made object to enter interstellar space, crossing the heliopause — the boundary where the Sun’s influence wanes and the interstellar medium begins. Voyager 1 is now providing invaluable data about this uncharted region.</p><p>Equipped with a Golden Record containing sounds and images representing Earth’s diversity, culture, and life, Voyager 1 serves as a message to any potential extraterrestrial intelligence. As of now, it continues to transmit data back to Earth, though its power sources are diminishing. Voyager 1’s ongoing mission and achievements remain a testament to human curiosity and the desire to explore the cosmos.</p><h4>What went wrong?</h4><p>In November, Voyager 1 suddenly stopped transmitting its usual stream of data containing information about the spacecraft’s health and measurements from its scientific instruments. Instead, the spacecraft’s data stream was entirely unintelligible. Because the telemetry was unreadable, experts on the ground could not easily tell what went wrong.</p><p>They hypothesized the source of the problem might be in the memory bank of the Flight Data System (FDS).</p><p>Given the immense distance between Earth and Voyager 1, the radio signals used to communicate between the two entities take ~23 hours to travel, meaning almost two days of wait to receive data back from the spacecraft after an initial emission! Debugging it would not be an easy task…</p><p>Through their investigation, Voyager’s ground team discovered a single chip responsible for storing a portion of the FDS memory stopped working, probably due to either a cosmic ray hit or a failure of aging hardware. This affected some of the computer’s software code.</p><blockquote>“That took out a section of memory, we have to relocate the code into a different portion of the memory, and then make sure that anything that uses those codes, those subroutines, know to go to the new location of memory, for access and to run it.” — Linda Spilker (early member of the program)</blockquote><h4>Fixing the problem</h4><p>Only about 3 percent of the FDS memory was corrupted by the bad chip, so engineers needed to transplant that code into another part of the memory bank. But no single location is large enough to hold the section of code in its entirety, NASA said.</p><p>So the Voyager team divided the code into sections for storage in different places in the FDS. This wasn’t just a copy-and-paste job. Engineers needed to modify some of the code to make sure it will all work together. “Any references to the location of that code in other parts of the FDS memory needed to be updated as well,” NASA said in a statement.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*hO_7_DAQkLMwXVwS.jpg" /><figcaption>A scanned 1970s-era photo of the Flight Data Subsystem computer aboard NASA’s Voyager spacecraft — NASA</figcaption></figure><p>Newer NASA missions benefit from having ground-based hardware and software simulators. These simulators allow engineers to thoroughly test new procedures before uploading commands to the actual spacecraft, ensuring they won’t cause any harm. However, Voyager, due to its age, lacks such ground simulators. Moreover, much of the mission’s original design documentation remains in paper form and hasn’t been digitized.</p><p>According to Spilker, examining the code was a strictly visual process. “We had to triple-check everything. Everyone scrutinized the code to ensure all connections were correct.”</p><p>After several weeks of preparation, NASA was prepared to upload a solution to restore Voyager 1’s Flight Data System (FDS), enabling it to resume packaging engineering data. This data stream includes crucial information about the spacecraft’s status, such as power levels and temperature measurements.</p><p><strong>The command was transmitted to Voyager 1 via one of NASA’s large Deep Space Network antennas on Thursday.</strong></p><p>Finally, on April 20th, 2024, confirmation arrived that the fix had succeeded. Voyager 1 resumed its vital task of collecting and transmitting data, providing ongoing insights from the distant reaches of our solar system.</p><p>Despite the challenges and the loss of scientific data during the repair process, Voyager 1 continues to inspire and contribute to our understanding of the cosmos, showcasing humanity’s enduring spirit of exploration.</p><p>In my opinion, this is a great featuring of what humans can achieve. Sending an object in the universe, keeping contact with it, and even fixing it after all these years and distance… It is truly amazing!</p><p>I hope you liked reading this paper as much as I liked writing it.<br>Make sure to check out my other articles, or follow my account to see my next publications, you might find something interesting!</p><p>As always, have a good day 🥳 🎊</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c0df9f17c5c4" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>