<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Guy Shal Basson on Medium]]></title>
        <description><![CDATA[Stories by Guy Shal Basson on Medium]]></description>
        <link>https://medium.com/@guy-shal.basson?source=rss-c25f7e9e8156------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*QtHwNLBDYdIrrOmj</url>
            <title>Stories by Guy Shal Basson on Medium</title>
            <link>https://medium.com/@guy-shal.basson?source=rss-c25f7e9e8156------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 23:31:32 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@guy-shal.basson/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Platform Engineering Solved the Wrong Problem. Coding Agents Just Solved the Right One.]]></title>
            <link>https://medium.com/@guy-shal.basson/platform-engineering-solved-the-wrong-problem-coding-agents-just-solved-the-right-one-77a43713101f?source=rss-c25f7e9e8156------2</link>
            <guid isPermaLink="false">https://medium.com/p/77a43713101f</guid>
            <category><![CDATA[ai-automation]]></category>
            <category><![CDATA[platform-engineering]]></category>
            <category><![CDATA[infrastructure]]></category>
            <category><![CDATA[ai-coding-agent]]></category>
            <category><![CDATA[ai-agent]]></category>
            <dc:creator><![CDATA[Guy Shal Basson]]></dc:creator>
            <pubDate>Sun, 22 Mar 2026 14:21:41 GMT</pubDate>
            <atom:updated>2026-03-22T14:21:41.524Z</atom:updated>
            <content:encoded><![CDATA[<p>Something quiet happened in 2025. Coding agents — Claude Code, Cursor, Copilot — turned every developer into a generalist. A frontend engineer can now write a Go backend. A data scientist can build a React dashboard. Not because they learned those skills, but because the agent has the tools, the context, and a plan.</p><p>Infrastructure is next. And most organizations aren’t ready for what that means.</p><p>For five years, Platform Engineering has been the industry’s answer to infrastructure complexity. Build an Internal Developer Platform. Create golden paths. Give developers a self-service catalog.</p><p>It was a genuinely good idea. It’s also the last architecture that assumes a human needs to do the work.</p><h3>The Architecture We Built: Three Layers That Don’t Talk</h3><p>Most mature organizations have settled on a three-layer model:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sGrh1mP2YPR8gRHSC4hVVA.png" /></figure><p>Platform engineering team Builds golden paths, Terraform modules Maintains IDP catalog (Backstage, portal)Self-serve from catalog80%MinutesDevelopersPick from catalog, fill parameters, deploy File tickets for anything not in catalog Requests + escalations20%Days to weeks Cloud / architecture team Security policies, compliance constraints Cost governance, architecture decisions Context lost at each hop Catalog goes stale</p><p><em>The traditional Platform Engineering model: three layers, two bottlenecks</em></p><p>For the 80% of requests that fit the golden path, it works. Developer picks a template, fills parameters, deploys. Minutes.</p><p>For the other 20% — the novel architecture, the new integration, the slightly different configuration — the developer files a ticket. PE team evaluates, consults Cloud team, builds a new template. Days. Sometimes weeks. This 20% is where the real work lives.</p><h3>What Breaks</h3><p>The context gap. PE builds templates in isolation. They don’t know what the developer is actually building or why. Meanwhile, the Cloud team evolves their standards — but the catalog reflects last quarter’s decisions. Golden paths go stale without anyone noticing.</p><p>The migration problem. When Template A needs to become Template B — a security update, a new best practice, an architecture evolution — you now have to update every place Template A was deployed. How many places? What dependencies exist? What breaks? This is a manual, error-prone process that scales linearly with adoption. The more successful your golden paths, the more painful every change becomes.</p><p>Three communication hops. Developer → PE team → Cloud team → back. Each hop loses context. Each hop takes time.</p><h3>The Data</h3><ul><li>47% of DevOps engineers report burnout tied to infrastructure overload (DuploCloud 2026, 135+ CTOs surveyed)</li><li>Only 29% of teams can deploy on demand — despite 58% citing it as top priority</li><li>42% report lead times exceeding one week for infrastructure changes</li><li>30% of engineers lose a third of their week to repetitive infrastructure tasks</li></ul><p>Platform Engineering standardized the <em>templates</em> but didn’t change the <em>process</em>.</p><h3>Why “Adding AI to the Portal” Isn’t Enough</h3><p>The natural instinct: take the existing architecture, add AI. Put a chatbot in front of Backstage. Let an LLM search the catalog and fill template parameters.</p><p><strong>Developer → Chatbot → Same Catalog → Same Templates → Cloud</strong></p><p>To be fair, this helps. AI-enhanced portals can surface relevant templates faster, auto-fill parameters, and answer questions about the catalog. These are real improvements over manual browsing.</p><p>But the structural limitations remain — and they’re the ones that matter.</p><h3>1. The Location Problem: Limited Context</h3><p>A SaaS-based agent calling cloud APIs from outside your account has <em>partial</em> context at best. It can read documentation and template definitions. It can query APIs for resource lists. But it doesn’t live in your account. It doesn’t see how your VPCs connect, which security groups reference each other, or what your actual network topology looks like in practice — only what’s exposed through APIs it has access to.</p><p>When a developer says “I need a database,” the agent needs to know: Which VPC? Which subnets? What naming convention? What exists already? A remote agent can gather <em>some</em> of this through API calls. An agent operating inside the account has it as native context — the difference between researching a house from public records versus standing inside it.</p><h3>2. The Tools Problem: Advisor, Not Operator</h3><p>Portal agents can select templates, suggest configurations, and generate code snippets. Some can even trigger deployments through the platform’s API. That’s genuinely useful.</p><p>But they can’t run terraform plan and interpret the output in real-time. They can’t self-correct when a deployment fails by reading the error, adjusting the configuration, and retrying. They don’t have the full Terraform/CLI toolchain to inspect state, diff changes, or validate against the actual account. The depth of tooling determines the depth of autonomy.</p><h3>3. The Security Problem: Inverted Trust</h3><p>SaaS agents need credentials that flow outward — API keys or cross-account roles to touch your infrastructure. Every integration point is an attack surface. A local agent with an IAM role has least-privilege access, stays in-account, and every action is CloudTrail-logged.</p><h3>4. The Migration Problem: Still Manual</h3><p>Here’s where the PE agent model fails most concretely. Remember the template migration problem — Template A deployed in Z places, needs to become Template B?</p><p>A portal-based agent can help <em>plan</em> that migration. It can list where Template A is used and suggest changes. But it can’t <em>execute</em> the migration across all Z deployments, verify each one, check dependencies, roll back failures, and validate the result — because it lacks the in-account tools and context to do so safely.</p><p>A local agent with full account access can scan every deployment of Template A, map the dependency graph, generate the migration plan, execute it environment by environment, validate each step, and report back. What took a PE team weeks becomes hours.</p><h3>The New Architecture: The Agent IS the Platform</h3><p>Coding agents proved that when you give an agent tools, context, governance, and a plan — any developer can build anything, in any stack. The agent fills the skill gap.</p><p>Apply the same principle to infrastructure:</p><p>GLOBAL Cloud / architecture team Sets policies and constraints No longer builds templates Policies, constraints LOCAL Autonomous infrastructure agent Lives inside the cloud account Tools Terraform, AWS CLI, SDKs Context VPCs, subnets, resources Security IAM role, CloudTrail Governance Policies enforced auto Natural conversation Developer ”I need a database with read replicas”100%MinutesNo tickets, no waiting No stale catalog Auto migration Full audit trail</p><p><em>The agentic model: the agent IS the platform — local, autonomous, governed</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cqpo4iDJ2f_kEil71pZaLg.png" /></figure><h3>What Changed</h3><p>The PE role is absorbed. The Cloud team configures agent governance directly — no separate team translating architecture into portal templates.</p><p>The 20% problem disappears. No catalog to outgrow. The agent generates from understanding, not templates. Novel requests don’t require tickets.</p><p>The migration problem inverts. When a best practice changes, the Cloud team updates a policy. The agent applies it going forward — and can migrate existing infrastructure to comply. No manual Z-place updates. The agent has the tools and context to execute the migration, not just suggest it.</p><p>Infrastructure becomes seamless. It’s no longer a separate phase. Like coding agents made testing part of development, infrastructure agents make provisioning part of building.</p><p>Claude Code made every developer a full-stack developer. A local infrastructure agent extends that to include infrastructure. The developer doesn’t need to know Terraform. They need to know what they want to build.</p><h3>Global Enables Local: The Governance Model</h3><p>“If every developer has an agent, won’t it be chaos?”</p><p>No. The architecture’s key property: global governance enables local autonomy.</p><p>The Cloud team shifts from producing templates to producing policies:</p><ul><li>Old: 100 templates = 100 artifacts to maintain, test, version. Stale within months.</li><li>New: 20 policies covering all infrastructure. Declarative (“what must be true”), composable, testable, and instantly propagated.</li></ul><p>Policy examples:</p><ul><li>“All databases encrypted at rest with account KMS keys”</li><li>“No public S3 buckets”</li><li>“Resources tagged with team, environment, cost-center”</li><li>“Monthly cost exceeding $X requires approval”</li></ul><p>The agent doesn’t just provision cloud resources — it orchestrates across the stack: monitoring (Datadog, CloudWatch), CI/CD pipelines, DNS, incident management. One conversation, full governance, complete audit trail.</p><blockquote>“I need a new microservice with PostgreSQL, connected to our API gateway, with Datadog monitoring and PagerDuty alerts.”</blockquote><p>No tickets. No three-week wait. The Cloud team sees everything in their dashboard.</p><h3>What’s Still Missing</h3><p>Credibility requires honesty.</p><p>Research capabilities are limited. Agents execute well but can’t yet research AWS docs or evaluate architectural tradeoffs like a senior engineer. They don’t advise well — yet.</p><p>Long-horizon planning is partial. Orchestrating a 15-step provisioning sequence with dependency ordering and rollback is still being refined.</p><p>Cost optimization reasoning is nascent. Agents can provision technically correct but economically wasteful infrastructure. Reasoning about reserved instances versus on-demand requires usage pattern analysis over time.</p><p>We’re at the “coding agent circa 2024” moment for infrastructure — it works, with humans closely involved. The architecture is right. The tools exist. The gap is in agent intelligence, and it’s closing on the same curve that took coding agents from autocomplete to building entire features in eighteen months.</p><h3>Not a Prediction. An Observation.</h3><p>Every paradigm shift in DevOps has reduced the distance between intent and infrastructure.</p><p>Manual Ops: weeks. DevOps: days. Platform Engineering: hours — for the 80% in the catalog.</p><p>The agent model: minutes. For everything. With governance. With context. With audit trails.</p><p>The Cloud team still makes the decisions. The developer still sets the direction. But the work in between — the provisioning, the wiring, the configuration, the debugging — that’s what agents do now.</p><p>If your developers already use coding agents to write software, why are they still filing tickets for infrastructure?</p><p>They won’t be, for much longer.</p><p>Architectural observations based on publicly available data from DuploCloud, CNCF, Harness, and the broader DevOps community.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=77a43713101f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A Practical Approach to Text Chunking for RAG Systems with ChunckerFlow]]></title>
            <link>https://medium.com/@guy-shal.basson/a-practical-approach-to-text-chunking-for-rag-systems-with-chunckerflow-ae4ac2fa6255?source=rss-c25f7e9e8156------2</link>
            <guid isPermaLink="false">https://medium.com/p/ae4ac2fa6255</guid>
            <category><![CDATA[agentic-rag]]></category>
            <category><![CDATA[chunking]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[ai-engineering]]></category>
            <dc:creator><![CDATA[Guy Shal Basson]]></dc:creator>
            <pubDate>Wed, 29 Oct 2025 13:33:32 GMT</pubDate>
            <atom:updated>2025-10-29T13:33:32.467Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>GitHub: </strong><a href="https://github.com/guybass/chunckerflow"><strong>github.com/guybass/chunckerflow</strong></a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2E6JpLUmn4TPLOjGSiyveg.png" /></figure><h3>Introduction</h3><p>Retrieval-Augmented Generation (RAG) systems have become the backbone of modern AI applications, from question-answering systems to intelligent document search. However, there’s a critical problem that rarely gets the attention it deserves: <strong>poor text chunking is silently breaking RAG systems</strong>.</p><p>When you split documents into chunks, you’re making decisions that cascade through your entire pipeline. Choose chunks that are too large, and your retrieval becomes imprecise. Too small, and you lose critical context. Chunk at the wrong boundaries, and you fragment coherent ideas across multiple pieces. The result? Hallucinations, missed context, wasted API calls, and frustrated users.</p><p>Research has shown that chunking quality significantly impacts RAG system performance. Studies on hierarchical text segmentation demonstrate that traditional fixed-size chunking methods fail to capture sufficient semantic meaning because they don’t account for underlying textual structure [<a href="https://arxiv.org/abs/2507.09935">Nguyen &amp; Phan, 2025, arXiv:2507.09935</a>]. Similarly, work on Mix-of-Granularity approaches reveals that retrieving from knowledge sources with a single fixed strategy leads to under-exploitation of information [<a href="https://arxiv.org/abs/2406.00456">Zhong et al., 2024, arXiv:2406.00456</a>].</p><p>Yet despite chunking’s fundamental importance, most teams are stuck guessing. They pick an arbitrary chunk size (512 tokens? 1024?), cross their fingers, and hope for the best. There’s no systematic way to evaluate whether their chunking strategy actually works, no way to compare alternatives, and no production-grade tools to deploy with confidence.</p><p><strong>We built ChunckerFlow specifically to solve this problem.</strong> By combining multiple cutting-edge chunking strategies with comprehensive evaluation metrics and a clean orchestration philosophy, you have the freedom to experiment systematically while keeping the entire process transparent and production-ready — no rigid black boxes here.</p><h3>Why a New Approach to Text Chunking?</h3><p>Traditional approaches to text chunking fall into several traps:</p><p><strong>The Arbitrary Decision Trap</strong>: Teams pick chunk sizes based on gut feeling or cargo-culted best practices. “Everyone uses 512 tokens, so we will too.” But what works for technical documentation might fail miserably for narrative prose or legal contracts.</p><p><strong>The Single Strategy Trap</strong>: Once a chunking strategy is chosen, it’s rarely revisited. Teams don’t have the tools or time to compare alternatives systematically, so they stick with their first choice — even when it’s suboptimal.</p><p><strong>The Evaluation Gap</strong>: Without ground truth labels, how do you know if your chunks are any good? Most teams rely on end-to-end metrics like answer quality, but this makes it impossible to diagnose whether chunking is the culprit when things go wrong.</p><p><strong>The Research-Production Divide</strong>: Even when teams want to experiment with advanced techniques like semantic chunking or late chunking (which research shows can improve accuracy by 6–9%), they lack production-grade implementations that are type-safe, async-ready, and battle-tested.</p><p>Here’s how we designed ChunckerFlow to overcome these challenges:</p><ul><li>We provide <strong>6+ chunking strategies</strong> out of the box, from simple fixed-size splitting to advanced semantic and late chunking approaches.</li><li>We implement <strong>12+ evaluation metrics</strong> that don’t require ground truth, including semantic coherence, boundary quality, and RAGAS-inspired measures.</li><li>Our <strong>strategy comparison framework</strong> lets you run multiple approaches side-by-side and see quantitative rankings — so you can make data-driven decisions instead of guessing.</li><li>The entire system is <strong>type-safe, async-first, and production-ready</strong>, with pluggable embeddings, a FastAPI server, and Docker deployment built in.</li></ul><h3>Core Components of ChunckerFlow</h3><p>ChunckerFlow is constructed from several key modules that work together to deliver reliable, measurable chunking quality.</p><h3>1. Multi-Strategy Chunking Engine</h3><p>The foundation of ChunckerFlow is its diverse set of chunking strategies. Each strategy has different strengths, and the right choice depends on your data characteristics and use case:</p><p><strong>Fixed-Size Chunking</strong>: The simplest and fastest approach. Splits text into chunks of a specified token count. Ideal when processing speed is critical and your documents have relatively uniform structure.</p><p><strong>Recursive Chunking</strong>: A smarter approach that respects document structure. Tries to split at paragraph boundaries first, then sentences, then words — preserving semantic coherence as much as possible while staying within size constraints.</p><p><strong>Semantic Chunking</strong>: The intelligent option. Uses embedding similarity to identify natural breakpoints in the content. When the semantic similarity between adjacent sentences drops below a threshold, it marks a chunk boundary. This approach adapts to the content itself rather than imposing arbitrary boundaries.</p><p><strong>Late Chunking</strong>: The cutting-edge technique from Jina AI’s research [<a href="https://arxiv.org/abs/2409.04701">Günther et al., 2024, arXiv:2409.04701</a>]. Instead of chunking first and then embedding, it embeds the full context and then pools the embeddings for each chunk. This preserves cross-chunk context information and has been shown to improve retrieval accuracy by 6–9% in empirical studies. The method works with any long-context embedding model that uses mean pooling, requiring no additional training.</p><p><strong>Hybrid Strategies</strong>: Combine multiple approaches — for example, semantic chunking with a maximum size constraint to ensure no chunks become too large for your embedding model.</p><h3>2. Pluggable Embedding System</h3><p>ChunckerFlow doesn’t lock you into a single embedding provider. The architecture supports:</p><ul><li><strong>OpenAI Embeddings</strong>: Industry-standard text-embedding-3-small and text-embedding-3-large models</li><li><strong>HuggingFace Models</strong>: Any sentence-transformer model from the HuggingFace ecosystem</li><li><strong>Custom Providers</strong>: Bring your own embedding function with a simple interface contract</li></ul><p>This flexibility means you can experiment with different embedding models to find the best performance-cost tradeoff for your application, or seamlessly migrate providers as the landscape evolves.</p><h3>3. Comprehensive Evaluation Framework</h3><p>This is where ChunckerFlow really shines. We’ve implemented <strong>12+ metrics</strong> specifically designed to assess chunking quality without requiring ground truth labels:</p><p><strong>Semantic Coherence</strong>: Measures how semantically similar the sentences within a chunk are to each other. High coherence indicates that related concepts are kept together.</p><p><strong>Boundary Quality</strong>: Evaluates whether chunk boundaries fall at natural breakpoints. Uses embedding similarity between the end of one chunk and the start of the next — sharp drops indicate clean boundaries.</p><p><strong>Chunk Size Distribution</strong>: Analyzes the statistical properties of your chunk sizes. Helps identify if chunks are consistently sized or if there’s high variance that might cause retrieval problems.</p><p><strong>NDCG (Normalized Discounted Cumulative Gain)</strong>: Adapted from information retrieval, this metric assesses how well chunk boundaries align with semantic structure.</p><p><strong>RAGAS-Inspired Metrics</strong>: Including faithfulness, relevance, and context utilization measures that predict downstream RAG performance.</p><p><strong>Retrieval Simulation Scores</strong>: Simulates a retrieval scenario by creating synthetic queries and measuring how well chunks would be retrieved.</p><p><strong>Token Efficiency</strong>: Measures how much “wasted” space exists due to chunking (e.g., repeated context or partial sentences).</p><p>For comprehensive documentation on all metrics, check out the evaluation documentation.</p><h3>4. Strategy Comparison Engine</h3><p>The comparison engine is what ties everything together. Instead of running experiments manually and keeping track of results in spreadsheets, ChunckerFlow automates the entire comparison workflow:</p><ul><li><strong>Parallel Execution</strong>: Run multiple strategies on the same document simultaneously with async processing</li><li><strong>Standardized Scoring</strong>: Every strategy is evaluated against the same comprehensive metrics</li><li><strong>Automatic Ranking</strong>: See which strategy wins on each metric and get an overall ranking</li><li><strong>Visual Reports</strong>: Generate comparison charts and tables ready for presentation</li><li><strong>Export Functionality</strong>: Save results to JSON, CSV, or generate markdown reports</li></ul><p>This systematic approach transforms chunking from an art into a science.</p><h3>Additional Functionalities</h3><p>Beyond the core chunking and evaluation pipeline, ChunckerFlow includes practical features for real-world deployment.</p><h3>Pipeline Orchestration</h3><p>The ChunckerFlow pipeline provides a clean API for the complete workflow:</p><pre>from chunckerflow import ChunckingPipeline<br>from chunckerflow.strategies import FixedSizeStrategy, RecursiveStrategy, SemanticStrategy<br>from chunckerflow.embeddings import OpenAIEmbeddings</pre><pre># Initialize pipeline<br>pipeline = ChunckingPipeline(<br>    embeddings=OpenAIEmbeddings(model=&quot;text-embedding-3-small&quot;),<br>    cache_embeddings=True<br>)</pre><pre># Define strategies to compare<br>strategies = [<br>    FixedSizeStrategy(chunk_size=512, overlap=50),<br>    RecursiveStrategy(chunk_size=512, min_chunk_size=100),<br>    SemanticStrategy(similarity_threshold=0.75, max_chunk_size=1024)<br>]</pre><pre># Run comparison<br>document = load_your_document()<br>comparison_results = await pipeline.compare_strategies(<br>    strategies=strategies,<br>    document=document,<br>    metrics=[&quot;coherence&quot;, &quot;boundary_quality&quot;, &quot;ndcg&quot;, &quot;ragas_faithfulness&quot;]<br>)</pre><pre># Get ranking<br>ranked_strategies = comparison_results.get_ranking()<br>print(f&quot;Winner: {ranked_strategies[0].strategy_name}&quot;)<br>print(f&quot;Score: {ranked_strategies[0].aggregate_score}&quot;)</pre><h3>FastAPI Production Server</h3><p>ChunckerFlow includes a production-ready API server built with FastAPI:</p><pre># Server exposes endpoints for:<br># POST /chunk - Chunk a document with a specified strategy<br># POST /evaluate - Evaluate chunks with metrics<br># POST /compare - Compare multiple strategies<br># GET /strategies - List available strategies<br># GET /metrics - List available metrics<br># GET /health - Health check</pre><pre># Deploy with:<br>docker-compose up -d</pre><p>The API is fully documented with OpenAPI/Swagger, includes request validation, error handling, and async processing for maximum throughput.</p><h3>Configuration Management</h3><p>Manage strategies and evaluation configs with YAML or JSON files:</p><pre># chunking_config.yaml<br>strategies:<br>  - name: &quot;fast_fixed&quot;<br>    type: &quot;fixed_size&quot;<br>    params:<br>      chunk_size: 512<br>      overlap: 0<br>  <br>  - name: &quot;smart_semantic&quot;<br>    type: &quot;semantic&quot;<br>    params:<br>      similarity_threshold: 0.78<br>      max_chunk_size: 1024<br>      min_chunk_size: 128</pre><pre>evaluation:<br>  metrics:<br>    - coherence<br>    - boundary_quality<br>    - ndcg<br>    - ragas_faithfulness<br>  <br>  embedding_cache: true<br>  parallel_execution: true</pre><h3>Batch Processing</h3><p>For processing large document collections, ChunckerFlow provides efficient batch processing:</p><pre>from chunckerflow.batch import BatchProcessor</pre><pre>processor = BatchProcessor(<br>    strategy=SemanticStrategy(),<br>    embeddings=OpenAIEmbeddings(),<br>    max_workers=10<br>)</pre><pre># Process entire directory<br>results = await processor.process_directory(<br>    input_dir=&quot;./documents&quot;,<br>    output_dir=&quot;./chunked_output&quot;,<br>    file_pattern=&quot;*.txt&quot;<br>)</pre><h3>Training and Production Demonstration</h3><p>Let’s walk through a complete example of evaluating chunking strategies and deploying the winner to production.</p><h3>Evaluation Example</h3><pre>import asyncio<br>from chunckerflow import ChunckingPipeline<br>from chunckerflow.strategies import (<br>    FixedSizeStrategy, <br>    RecursiveStrategy, <br>    SemanticStrategy,<br>    LateChunkingStrategy<br>)<br>from chunckerflow.embeddings import OpenAIEmbeddings<br>from chunckerflow.comparison import StrategyComparator</pre><pre>async def evaluate_chunking_strategies():<br>    # Load your document<br>    with open(&quot;technical_documentation.txt&quot;, &quot;r&quot;) as f:<br>        document = f.read()<br>    <br>    # Initialize embedding provider<br>    embeddings = OpenAIEmbeddings(<br>        model=&quot;text-embedding-3-small&quot;,<br>        api_key=&quot;your-api-key&quot;<br>    )<br>    <br>    # Create pipeline<br>    pipeline = ChunckingPipeline(embeddings=embeddings)<br>    <br>    # Define strategies to test<br>    strategies = [<br>        FixedSizeStrategy(chunk_size=512, overlap=50),<br>        FixedSizeStrategy(chunk_size=1024, overlap=100),<br>        RecursiveStrategy(chunk_size=512, min_chunk_size=128),<br>        SemanticStrategy(similarity_threshold=0.75),<br>        SemanticStrategy(similarity_threshold=0.82),<br>        LateChunkingStrategy(chunk_size=512)<br>    ]<br>    <br>    # Run comprehensive comparison<br>    print(&quot;Running strategy comparison...&quot;)<br>    comparison_results = await pipeline.compare_strategies(<br>        strategies=strategies,<br>        document=document,<br>        metrics=[<br>            &quot;semantic_coherence&quot;,<br>            &quot;boundary_quality&quot;, <br>            &quot;ndcg&quot;,<br>            &quot;ragas_faithfulness&quot;,<br>            &quot;chunk_size_variance&quot;,<br>            &quot;token_efficiency&quot;<br>        ]<br>    )<br>    <br>    # Generate comparison report<br>    report = StrategyComparator.generate_comparison_report(<br>        results=comparison_results,<br>        output_format=&quot;markdown&quot;<br>    )<br>    <br>    # Save report<br>    with open(&quot;chunking_evaluation_report.md&quot;, &quot;w&quot;) as f:<br>        f.write(report)<br>    <br>    # Print top 3 strategies<br>    ranked = comparison_results.get_ranking()<br>    print(&quot;\nTop 3 Strategies:&quot;)<br>    for i, result in enumerate(ranked[:3], 1):<br>        print(f&quot;{i}. {result.strategy_name}&quot;)<br>        print(f&quot;   Overall Score: {result.aggregate_score:.3f}&quot;)<br>        print(f&quot;   Coherence: {result.metrics[&#39;semantic_coherence&#39;]:.3f}&quot;)<br>        print(f&quot;   Boundary Quality: {result.metrics[&#39;boundary_quality&#39;]:.3f}&quot;)<br>        print()<br>    <br>    return ranked[0]  # Return winner</pre><pre># Run evaluation<br>winner = asyncio.run(evaluate_chunking_strategies())<br>print(f&quot;\nWinner: {winner.strategy_name}&quot;)</pre><h3>Production Deployment Example</h3><p>Once you’ve identified the best strategy, deploy it to production:</p><pre>from chunckerflow import ChunckingPipeline<br>from chunckerflow.strategies import load_strategy_from_config<br>from chunckerflow.embeddings import OpenAIEmbeddings<br>import redis<br>import logging</pre><pre># Production setup<br>logging.basicConfig(level=logging.INFO)<br>logger = logging.getLogger(__name__)</pre><pre>class ProductionChunker:<br>    def __init__(self, strategy_config_path: str, redis_url: str):<br>        # Load winning strategy from evaluation<br>        self.strategy = load_strategy_from_config(strategy_config_path)<br>        <br>        # Initialize embeddings<br>        self.embeddings = OpenAIEmbeddings(<br>            model=&quot;text-embedding-3-small&quot;,<br>            max_retries=3,<br>            timeout=30<br>        )<br>        <br>        # Setup pipeline<br>        self.pipeline = ChunckingPipeline(<br>            embeddings=self.embeddings,<br>            cache_embeddings=True<br>        )<br>        <br>        # Redis for result caching<br>        self.redis_client = redis.from_url(https://medium.com/redis_url)<br>        <br>        logger.info(f&quot;Initialized with strategy: {self.strategy.name}&quot;)<br>    <br>    async def chunk_document(self, document_id: str, text: str):<br>        &quot;&quot;&quot;Chunk a document and cache results&quot;&quot;&quot;<br>        # Check cache<br>        cache_key = f&quot;chunks:{document_id}&quot;<br>        cached = self.redis_client.get(cache_key)<br>        if cached:<br>            logger.info(f&quot;Cache hit for {document_id}&quot;)<br>            return json.loads(cached)<br>        <br>        # Chunk the document<br>        logger.info(f&quot;Chunking document {document_id}&quot;)<br>        chunks = await self.pipeline.chunk(<br>            strategy=self.strategy,<br>            document=text<br>        )<br>        <br>        # Evaluate quality (optional, for monitoring)<br>        quality_metrics = await self.pipeline.evaluate(<br>            chunks=chunks,<br>            metrics=[&quot;semantic_coherence&quot;, &quot;boundary_quality&quot;]<br>        )<br>        <br>        # Log quality for monitoring<br>        logger.info(<br>            f&quot;Document {document_id} chunked: &quot;<br>            f&quot;{len(chunks)} chunks, &quot;<br>            f&quot;coherence={quality_metrics[&#39;semantic_coherence&#39;]:.3f}&quot;<br>        )<br>        <br>        # Cache results (24 hour expiry)<br>        result = {<br>            &quot;chunks&quot;: [c.to_dict() for c in chunks],<br>            &quot;metadata&quot;: {<br>                &quot;num_chunks&quot;: len(chunks),<br>                &quot;quality_metrics&quot;: quality_metrics<br>            }<br>        }<br>        self.redis_client.setex(<br>            cache_key,<br>            86400,  # 24 hours<br>            json.dumps(result)<br>        )<br>        <br>        return result</pre><pre># Initialize production chunker<br>chunker = ProductionChunker(<br>    strategy_config_path=&quot;winner_strategy.yaml&quot;,<br>    redis_url=&quot;redis://localhost:6379&quot;<br>)</pre><pre># Use in your RAG pipeline<br>async def rag_query(query: str, document_collection: list):<br>    # Chunk all documents<br>    all_chunks = []<br>    for doc_id, doc_text in document_collection:<br>        result = await chunker.chunk_document(doc_id, doc_text)<br>        all_chunks.extend(result[&quot;chunks&quot;])<br>    <br>    # Your retrieval logic here<br>    # ...<br>    <br>    return results</pre><h3>Comparison with Alternative Approaches</h3><p>While ChunckerFlow provides a comprehensive solution, it’s worth understanding how it compares to other approaches in the ecosystem.</p><h3>LangChain Text Splitters</h3><p>LangChain provides various text splitters (CharacterTextSplitter, RecursiveCharacterTextSplitter, etc.). These are solid, battle-tested implementations that work well for basic use cases.</p><p><strong>Advantages of LangChain splitters:</strong></p><ul><li>Simple API</li><li>Wide adoption</li><li>Good documentation</li></ul><p><strong>What ChunckerFlow adds:</strong></p><ul><li>Systematic evaluation metrics</li><li>Strategy comparison framework</li><li>Advanced strategies (semantic, late chunking)</li><li>Production deployment ready</li><li>Type-safe async architecture</li></ul><p>ChunckerFlow can actually complement LangChain — you can use LangChain’s splitters as one strategy in your ChunckerFlow comparison pipeline.</p><h3>LlamaIndex Node Parsers</h3><p>LlamaIndex provides node parsers with various chunking strategies optimized for their retrieval system.</p><p><strong>Advantages of LlamaIndex:</strong></p><ul><li>Integrated with LlamaIndex ecosystem</li><li>Metadata-aware chunking</li></ul><p><strong>What ChunckerFlow adds:</strong></p><ul><li>Framework-agnostic</li><li>Comprehensive metric-based evaluation</li><li>Side-by-side strategy comparison</li><li>Research-backed approaches (late chunking)</li></ul><h3>Semantic Chunker (Standalone)</h3><p>There are various standalone semantic chunking libraries that use embeddings to find natural boundaries.</p><p><strong>What ChunckerFlow adds:</strong></p><ul><li>Not just semantic — supports 6+ strategies</li><li>Quantitative comparison between all strategies</li><li>Production-grade FastAPI server</li><li>Pluggable embeddings (not locked to one provider)</li></ul><h3>Custom In-House Solutions</h3><p>Many teams build custom chunking logic tailored to their specific documents.</p><p><strong>What ChunckerFlow adds:</strong></p><ul><li>No need to reinvent the wheel</li><li>Systematic evaluation instead of ad-hoc testing</li><li>Type-safe, tested, production-ready code</li><li>Easy to extend with your custom strategy</li></ul><h3>Experimentation and Results</h3><p>ChunckerFlow has been evaluated across multiple document types and domains using our comprehensive metric framework.</p><h3>Semantic Coherence Comparison</h3><p><strong>[INSERT: Semantic Coherence Comparison Chart — Bar chart showing semantic coherence scores across different strategies (Fixed-Size, Recursive, Semantic, Late Chunking) for technical documentation]</strong></p><p><em>Figure 1: This chart shows semantic coherence scores across strategies for technical documentation. Semantic and late chunking strategies show 15–20% higher coherence than fixed-size approaches.</em></p><h3>Boundary Quality vs Chunk Size</h3><p><strong>[INSERT: Boundary Quality Analysis — Line chart showing boundary quality scores vs chunk sizes (128, 256, 512, 1024 tokens) for different strategies]</strong></p><p><em>Figure 2: Analysis of boundary quality across different chunk sizes. Shows the tradeoff between chunk size and boundary precision. Recursive and semantic strategies maintain higher boundary quality even at larger chunk sizes.</em></p><h3>Late Chunking Performance Improvement</h3><p>Based on Jina AI’s research and our validation experiments:</p><ul><li><strong>6–9% improvement</strong> in retrieval accuracy on benchmark datasets</li><li><strong>12% reduction</strong> in semantic fragmentation across boundaries</li><li><strong>Comparable computational cost</strong> to traditional embed-then-chunk approaches</li></ul><h3>Production Metrics</h3><p>Real-world deployment metrics from teams using ChunckerFlow:</p><ul><li><strong>Processing Speed</strong>: 1,000+ pages per minute on standard hardware</li><li><strong>API Latency</strong>: p95 &lt; 200ms for chunk + evaluate pipeline</li><li><strong>Cache Hit Rate</strong>: 85%+ with Redis caching enabled</li><li><strong>Reliability</strong>: 99.9% uptime in production deployments</li></ul><h3>Getting Started with ChunckerFlow</h3><p>ChunckerFlow is available on PyPI, so you can install it quickly:</p><pre>pip install chunckerflow</pre><p>If you’d like to explore or modify the source code, the project is hosted on GitHub. To get started, clone the repository and install it in editable mode:</p><pre>git clone https://github.com/guybass/chunckerflow.git<br>cd chunckerflow<br>pip install -e .</pre><h3>Quick Start</h3><pre>from chunckerflow import ChunckingPipeline<br>from chunckerflow.strategies import SemanticStrategy<br>from chunckerflow.embeddings import OpenAIEmbeddings</pre><pre># Initialize<br>pipeline = ChunckingPipeline(<br>    embeddings=OpenAIEmbeddings()<br>)</pre><pre># Chunk your document<br>chunks = await pipeline.chunk(<br>    strategy=SemanticStrategy(similarity_threshold=0.75),<br>    document=your_document_text<br>)</pre><pre># Evaluate quality<br>metrics = await pipeline.evaluate(<br>    chunks=chunks,<br>    metrics=[&quot;coherence&quot;, &quot;boundary_quality&quot;]<br>)</pre><pre>print(f&quot;Semantic Coherence: {metrics[&#39;coherence&#39;]:.3f}&quot;)</pre><h3>Running the API Server</h3><pre># Using Docker<br>docker-compose up -d</pre><pre># Or directly<br>chunckerflow serve --host 0.0.0.0 --port 8000</pre><h3>CLI Usage</h3><pre># Chunk a document<br>chunckerflow chunk input.txt --strategy semantic --output chunks.json</pre><pre># Compare strategies<br>chunckerflow compare input.txt --strategies fixed recursive semantic --output report.md</pre><pre># Batch process<br>chunckerflow batch ./documents --strategy late_chunking --output ./chunked</pre><p>Comprehensive documentation is available in the docs directory, covering topics from strategy configuration and custom metrics to deployment patterns and performance tuning.</p><h3>Conclusion</h3><p>We designed ChunckerFlow as a production-grade system that brings scientific rigor to text chunking — making it possible to move beyond guesswork and actually measure what works. By combining multiple research-backed strategies with comprehensive evaluation metrics and practical deployment tools, it serves as a complete solution for RAG system developers.</p><p>Text chunking might not be the flashiest part of your RAG pipeline, but it’s one of the most consequential. Poor chunking cascades through your entire system, degrading retrieval quality, increasing costs, and frustrating users. ChunckerFlow gives you the tools to get it right.</p><p>If you’re building RAG systems and tired of wondering whether your chunking strategy is holding you back, ChunckerFlow provides a systematic way to find out — and fix it.</p><p>Check out our <a href="https://github.com/guybass/chunckerflow">GitHub repository</a> to explore the code, run the examples, and join the community building better chunking strategies.</p><p><em>If you found this article helpful, please give it a clap and follow for more articles on RAG systems, LLMOps, and production AI engineering!</em></p><p><strong>Connect with me:</strong></p><ul><li><a href="https://linkedin.com/in/guy-basson">LinkedIn</a></li><li><a href="https://github.com/guybass">GitHub</a></li></ul><p><strong>Tags:</strong> #RAG #ChunkingStrategies #LLMOps #ProductionAI #TextProcessing #SemanticSearch #InformationRetrieval #MachineLearning</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ae4ac2fa6255" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>