<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by TinyTwo on Medium]]></title>
        <description><![CDATA[Stories by TinyTwo on Medium]]></description>
        <link>https://medium.com/@guqung6?source=rss-e1958981e79b------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*G0oJogUF8nY71hAj.jpg</url>
            <title>Stories by TinyTwo on Medium</title>
            <link>https://medium.com/@guqung6?source=rss-e1958981e79b------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 15:45:27 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@guqung6/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Introducing TT Notebook: An AI Reading Assistant Built on Everything Above]]></title>
            <link>https://medium.com/@guqung6/introducing-tt-notebook-an-ai-reading-assistant-built-on-everything-above-4bcfbbbd4653?source=rss-e1958981e79b------2</link>
            <guid isPermaLink="false">https://medium.com/p/4bcfbbbd4653</guid>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[swiftui]]></category>
            <category><![CDATA[macos]]></category>
            <category><![CDATA[reading]]></category>
            <dc:creator><![CDATA[TinyTwo]]></dc:creator>
            <pubDate>Mon, 25 May 2026 13:01:02 GMT</pubDate>
            <atom:updated>2026-05-25T13:01:02.331Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*U-r0m6zdboXEo3iiToyzlA.jpeg" /></figure><p><em>Part 6 of “Building AI Agents That Actually Work” — a series on production-grade AI engineering.</em></p><p>Over the past five articles, we’ve built a complete picture of what production-grade AI engineering looks like:</p><ul><li><strong>Part 1:</strong> Why naive RAG fails — token explosion, imprecise retrieval, conversation collapse</li><li><strong>Part 2:</strong> Context assembly — the 8-step pipeline that orchestrates document retrieval, annotation selection, and history compression</li><li><strong>Part 3:</strong> Smart caching — three tiers from memory to API that cut latency by 100x and costs by 90%</li><li><strong>Part 4:</strong> Conversation memory — hierarchical summarization that makes 100-message conversations work as well as 10-message ones</li><li><strong>Part 5:</strong> Embedding architecture — on-device vs. cloud, and why user choice is a competitive advantage</li><li>We didn’t build these systems in a vacuum. They power a real product.</li><li>Today, I’m introducing <strong>TT Notebook</strong> — a native macOS reading assistant that combines professional PDF annotation with AI-powered discussion and automatic note generation. Everything described in this series is running in production, handling documents from 5-page papers to 300-page textbooks.</li></ul><h3>The Problem</h3><p>If you read seriously — research papers, technical documentation, long-form reports — you’ve probably experienced these frustrations:</p><p><strong>Context-switching kills flow.</strong> You’re deep in a paper, encounter an unfamiliar concept, and have to switch to a browser or ChatGPT to look it up. By the time you get back, you’ve lost your place and your train of thought.</p><p><strong>Notes are scattered everywhere.</strong> Highlights in one app, notes in another, AI conversations in a third. When you want to synthesize what you’ve learned, you’re hunting across tools.</p><p><strong>Existing tools force a choice.</strong> PDF Expert gives you excellent reading and annotation but no AI. NotebookLM gives you AI discussion but no direct PDF annotation. You can’t get both in one place.</p><p>TT Notebook eliminates these trade-offs. Read, annotate, discuss with AI, and generate structured notes — all without leaving your document.</p><h3>Core Features</h3><h3>Professional PDF Reading &amp; Annotation</h3><p>TT Notebook is a native macOS app built with SwiftUI and PDFKit. This isn’t a web wrapper or Electron app — it’s a proper Mac application with native performance and system integration.</p><ul><li><strong>4-color highlighting</strong> (yellow, green, blue, pink) with multi-line selection support</li><li><strong>Rich annotation notes</strong> with Markdown formatting and inline images</li><li><strong>Bidirectional annotation linking</strong> — connect related highlights across different pages</li><li><strong>Reading progress tracking</strong> and bookmark navigation</li><li><strong>Annotation sidebar</strong> with text preview and click-to-navigate</li></ul><h3>Instant AI Explanations</h3><p>Select any text in your document, and a floating toolbar appears. One tap sends the selected passage to the AI with surrounding context. The explanation appears in the chat panel — no tab switching, no copy-pasting, no context loss.</p><p>The AI doesn’t just see the selected text. Behind the scenes, the context assembly pipeline (Part 2) pulls in relevant annotations, nearby content, and conversation history to give a contextually aware answer.</p><h3>Document-Level AI Discussion</h3><p>The right panel hosts a full conversation interface for discussing the entire document. Ask questions, explore concepts, challenge arguments — the AI has access to the complete document through our retrieval system.</p><p>Key capabilities:</p><ul><li><strong>Streaming responses</strong> — text appears in real-time, not as a batch after processing</li><li><strong>Multi-provider support</strong> — Claude (Anthropic), OpenAI, and Doubao (ByteDance)</li><li><strong>Conversation persistence</strong> — every chat is saved per document and survives app restarts</li><li><strong>Structural awareness</strong> — ask “What does Chapter 5 say about X?” and the AI targets that specific section</li></ul><h3>Smart Note Generation</h3><p>After reading and annotating a document, TT Notebook can automatically generate structured reading notes from your accumulated highlights, personal notes, and AI conversation history.</p><p>The generated notes follow a structured format:</p><ul><li><strong>Summary</strong> — overview of the document</li><li><strong>Key Points</strong> — main arguments and findings</li><li><strong>Core Concepts</strong> — important terms and ideas explained</li><li><strong>Personal Insights</strong> — synthesized from your annotation notes</li><li><strong>Questions to Explore</strong> — from your AI conversations</li><li><strong>Selected Quotes</strong> — from your highlights</li><li>These aren’t generic summaries — they’re personalized to your reading experience, incorporating what you highlighted, what you wrote, and what you discussed with the AI.</li></ul><h3>Document Organization</h3><p>Manage your reading library with folders, search, and metadata tracking. Each document shows its reading progress, annotation count, and last-accessed date.</p><h3>How It Compares</h3><p>CapabilityPDF ExpertMarginNoteNotebookLMTT NotebookPDF reading qualityExcellentGoodBasicExcellentAnnotation systemGoodExcellentNoneExcellentAI conversationNoneNoneExcellentExcellentAuto note generationNoneNoneBasicExcellentPrivacy / local-firstExcellentGoodNone (cloud-only)ExcellentNative macOSYesYesNo (web)YesWorks offlineYesYesNoYes</p><p>TT Notebook is the first tool that combines professional-grade PDF reading with deep AI integration — not as an afterthought, but as a core design principle.</p><h3>Privacy &amp; User Control</h3><p>Your data stays yours. Here’s how:</p><ul><li><strong>PDFs stay local.</strong> Documents are never uploaded to any server. The app uses macOS Security-Scoped Bookmarks to access files in their original location.</li><li><strong>You own your API keys.</strong> There’s no middleman server. Your API calls go directly from your Mac to the provider (Claude, OpenAI, or Doubao). We never see your conversations or documents.</li><li><strong>On-device embeddings available.</strong> Choose Apple’s Core ML embeddings for zero-cost, fully offline vector search. No data leaves your device.</li><li><strong>Optional iCloud sync.</strong> Premium users can sync across devices via CloudKit, but it’s opt-in — local-only mode is always available.</li></ul><h3>The Technology Inside</h3><p>Every technical component described in this series runs in TT Notebook:</p><p><strong>Context Assembly Pipeline</strong> (→ <a href="https://medium.com/link">Part 2</a>) When you ask a question, the 8-step pipeline assembles the optimal context: relevant document chunks, your annotations, your notes, and conversation history — all within your provider’s token budget.</p><p><strong>Three-Tier Caching</strong> (→ <a href="https://medium.com/link">Part 3</a>) First question takes a few seconds (cold start). Every subsequent question runs in under 200ms. Prompt Cache cuts API costs by 90% for repeated conversations about the same document.</p><p><strong>Hierarchical Conversation Memory</strong> (→ <a href="https://medium.com/link">Part 4</a>) Discuss a document for 100+ messages without degradation. The tiered summarization system preserves your full conversation arc while keeping token usage constant.</p><p><strong>Multi-Provider Embeddings</strong> (→ <a href="https://medium.com/link">Part 5</a>) Choose between Apple on-device (free, private, offline), OpenAI (highest quality), or Doubao (China-optimized). Switch providers anytime — caches update automatically.</p><h3>Pricing</h3><p><strong>Free tier:</strong></p><ul><li>15 documents</li><li>Full reading, annotation, and AI features</li><li>Local storage only</li><li>Apple on-device embeddings included</li><li><strong>Premium (one-time purchase, not a subscription):</strong></li><li>Unlimited documents</li><li>iCloud sync across devices</li><li>Export notes as PDF and Markdown</li><li>Print capabilities</li><li>All AI features (same providers, same quality)</li><li>We chose a one-time purchase model deliberately. Researchers and students shouldn’t face recurring charges for a reading tool. Your own API keys power the AI features — there are no ongoing costs from us.</li></ul><h3>Platform</h3><ul><li><strong>macOS 13+</strong> — native Swift/SwiftUI application</li><li><strong>Languages</strong> — English and Chinese (Simplified)</li><li><strong>AI Providers</strong> — Claude, OpenAI, Doubao (bring your own API key)</li><li><strong>Embedding Providers</strong> — Apple on-device, OpenAI, Doubao</li></ul><h3>What’s Next</h3><p>TT Notebook is actively developed. On the roadmap:</p><ul><li><strong>Multi-document knowledge base</strong> — search and discuss across your entire library</li><li><strong>Word and Markdown support</strong> — beyond PDF</li><li><strong>iOS companion app</strong> — read on iPad, sync with Mac</li></ul><h3>Get Started</h3><p>TT Notebook is available on the Mac App Store.</p><p><a href="https://apps.apple.com/cn/app/tt-notebook/id6758238388?mt=12">Download TT Notebook →</a></p><p>Have feature requests, technical questions, or want to discuss AI reading workflows? Reach me at <a href="/service/mailto:support@tinytwo.me"><strong>support@tinytwo.me</strong></a> — I read every email.</p><h3>Series Index</h3><ol><li><a href="https://medium.com/link">Why Most RAG Implementations Fail — and What to Do Instead</a></li><li><a href="https://medium.com/link">Context Assembly: The Missing Layer Between Your LLM and Your Data</a></li><li><a href="https://medium.com/link">Smart Caching for LLM Apps: Three Tiers from Memory to API</a></li><li><a href="https://medium.com/link">Conversation Memory That Scales: Hierarchical Summarization for Long Chats</a></li><li><a href="https://medium.com/link">On-Device vs. Cloud Embeddings: Building Privacy-First AI Apps</a></li><li><strong>Introducing TT Notebook</strong> (you are here)</li></ol><p><em>Thank you for following this series. Building TT Notebook has been a deep exploration of what it takes to make AI systems work reliably in production. I hope these articles help you build better AI applications — and that TT Notebook helps you read better.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4bcfbbbd4653" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[On-Device vs. Cloud Embeddings: Building Privacy-First AI Apps]]></title>
            <link>https://medium.com/@guqung6/on-device-vs-cloud-embeddings-building-privacy-first-ai-apps-32c488be080f?source=rss-e1958981e79b------2</link>
            <guid isPermaLink="false">https://medium.com/p/32c488be080f</guid>
            <category><![CDATA[coreml]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[embedding]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[vector-database]]></category>
            <dc:creator><![CDATA[TinyTwo]]></dc:creator>
            <pubDate>Mon, 18 May 2026 13:01:00 GMT</pubDate>
            <atom:updated>2026-05-18T13:01:00.798Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DQIHdEKB5AbTcY-QSrfyvQ.jpeg" /></figure><p><em>Part 5 of “Building AI Agents That Actually Work” — a series on production-grade AI engineering.</em></p><p>Throughout this series, we’ve talked about embeddings as if they’re a solved problem — generate vectors, cache them, use them for retrieval. But there’s a question we haven’t addressed: where do those vectors come from?</p><p>You have two fundamentally different options: send your data to a cloud API and get high-quality embeddings back, or run a model locally on the user’s device and keep everything private. Each choice cascades through your entire architecture — affecting retrieval quality, latency, cost, offline capability, and user trust.</p><p>In this article, I’ll walk through the three embedding providers we support in our reading assistant, the trade-offs between them, and why giving users a choice turned out to be a competitive advantage.</p><h3>The Three Providers</h3><p>ProviderModelDimensionsMax InputLatencyCostOfflineApple (on-device)paraphrase-multilingual-MiniLM-L12-v2384128 tokens~100msFreeYesOpenAItext-embedding-3-small15368191 tokens~200ms/batchPer-tokenNoDoubao (ByteDance)Provider-specificVariableVariable~200ms/batchPer-tokenNo</p><p>These aren’t interchangeable. Each represents a fundamentally different trade-off.</p><h3>Apple On-Device Embeddings</h3><h3>How It Works</h3><p>Apple’s approach runs a Core ML model directly on the user’s Mac, using the CPU and Neural Engine. No network calls, no API keys, no data leaves the device.</p><p>The model — paraphrase-multilingual-MiniLM-L12-v2 — is a multilingual sentence transformer that produces 384-dimensional vectors. It’s small (~50MB) but capable, supporting 50+ languages including English and Chinese.</p><h3>The Advantages</h3><p><strong>Zero cost.</strong> No API calls means no token charges. A user can process 500 documents without spending a cent on embeddings.</p><p><strong>Zero latency for generation.</strong> No network round-trip. Generation takes ~100ms per text on the Neural Engine, and results are cached for future use.</p><p><strong>Full offline capability.</strong> The user can work on a plane, in a cafe with bad WiFi, or in an environment where sending documents to external servers isn’t allowed.</p><p><strong>Privacy by default.</strong> This is the big one. Academic researchers handling unpublished papers, lawyers reviewing confidential documents, journalists protecting sources — these users need a guarantee that their documents never leave their device. On-device embeddings provide that guarantee architecturally, not just through policy.</p><h3>The Trade-offs</h3><p><strong>Lower dimensionality (384 vs. 1536).</strong> Fewer dimensions means less nuanced semantic representation. For most reading assistant queries, this is adequate. For highly specialized or technical content, cloud embeddings produce better retrieval results.</p><p><strong>Shorter max input (128 tokens).</strong> The model truncates inputs longer than 128 tokens. For document chunks (which we size at 2,000–3,000 tokens), only the beginning is embedded. This biases retrieval toward the opening of each chunk. In practice, this is acceptable because chunk openings typically contain topic sentences.</p><p><strong>No cross-query learning.</strong> Cloud providers can improve their models based on aggregate usage patterns. On-device models are static — they improve only when the app ships a new model version.</p><h3>OpenAI Cloud Embeddings</h3><h3>How It Works</h3><p>The app sends text to OpenAI’s embedding API and receives 1536-dimensional vectors back. This requires an API key and network connectivity.</p><h3>The Advantages</h3><p><strong>Higher semantic quality.</strong> 1536 dimensions capture more nuanced relationships between concepts. For technical documents where subtle distinctions matter (e.g., “gradient descent” vs. “stochastic gradient descent”), cloud embeddings produce measurably better retrieval.</p><p><strong>Longer input support (8191 tokens).</strong> Each chunk can be fully represented, not truncated. This means embeddings capture the complete semantic content of each passage.</p><p><strong>Continuous improvement.</strong> OpenAI periodically updates their embedding models with better training and larger datasets.</p><h3>The Trade-offs</h3><p><strong>Cost.</strong> Every embedding generation costs money. For a 200-page document with 100 chunks, that’s 100 API calls. Our caching system (described in <a href="https://medium.com/link-to-part-3">Part 3</a>) mitigates this — embeddings are generated once and cached — but the initial cost exists.</p><p><strong>Latency.</strong> Network round-trips add 200ms+ per batch. For initial document processing, this means 10–30 seconds depending on document size. Our batch processing system sends up to 100 texts per API call with 100ms inter-batch delays to avoid rate limits.</p><p><strong>Privacy implications.</strong> Document content is sent to OpenAI’s servers. For many use cases this is fine. For sensitive documents, it’s a deal-breaker.</p><p><strong>Offline unavailable.</strong> No network, no embeddings. The system degrades to proximity-based retrieval without cached embeddings.</p><h3>Doubao (ByteDance) Embeddings</h3><h3>Why a Third Provider?</h3><p>Users in mainland China face a practical problem: OpenAI’s API is unreliable or blocked behind a VPN. Doubao provides a direct-connection alternative optimized for the Chinese market.</p><p>The trade-offs mirror OpenAI’s — cloud-based, API-key required, network dependent — but with a critical advantage for Chinese users: reliable access without proxy configuration.</p><h3>The Privacy-Quality-Cost Triangle</h3><p>Every embedding decision involves three competing concerns:</p><pre>         Privacy<br>        /      \<br>       /        \<br>      /          \<br>   Quality ─── Cost</pre><ul><li><strong>Maximize privacy</strong> → On-device (Apple). Quality is adequate, cost is zero, but dimensions are limited.</li><li><strong>Maximize quality</strong> → Cloud (OpenAI). Privacy is reduced, cost is non-zero, but retrieval is best.</li><li><strong>Minimize cost</strong> → On-device (Apple). Quality trade-off is acceptable, privacy is a bonus.</li><li>No single option wins on all three axes. That’s why we let users choose.</li></ul><h3>Incremental Embedding Updates</h3><p>Regardless of provider, regenerating all embeddings every time something changes is wasteful. We implement incremental updates for annotations and notes.</p><h3>The Trigger Mechanism</h3><p>When a user creates or edits an annotation, the view posts a notification:</p><pre>NotificationCenter → .annotationDidChange → MainView listener</pre><p>The listener collects all annotations for the document and passes them to the embedding service, which compares timestamps:</p><ul><li><strong>Annotation unchanged</strong> (updatedAt ≤ cached timestamp) → skip</li><li><strong>Annotation modified</strong> (updatedAt &gt; cached timestamp) → regenerate only this embedding</li><li><strong>New annotation</strong> → generate and append</li><li>The same pattern applies to notes via .noteDidChange.</li></ul><h3>Why This Matters</h3><p>Without incremental updates, adding a single annotation to a document with 50 existing annotations would regenerate all 50 embeddings. With OpenAI, that’s 50 unnecessary API calls. With Apple on-device, it’s 50 × 100ms = 5 seconds of unnecessary computation.</p><p>Incremental updates reduce this to a single embedding generation — regardless of how many existing embeddings are cached.</p><h3>Cache Validation Across Providers</h3><p>When a user switches embedding providers, all cached embeddings become invalid. Vectors from different embedding spaces can’t be compared — cosine similarity between an Apple 384-dimensional vector and an OpenAI 1536-dimensional vector is mathematically meaningless.</p><p>Our three-dimensional cache validation (described in <a href="https://medium.com/link-to-part-3">Part 3</a>) catches this automatically:</p><ol><li><strong>Page count check</strong> — did the document change?</li><li><strong>Model name check</strong> — is this the same model?</li><li><strong>Provider check</strong> — is this the same embedding service?</li><li>If the user switches from Apple to OpenAI, the provider check fails on next access, and the system regenerates all embeddings using the new provider. The transition is seamless — the user doesn’t need to manually clear caches.</li></ol><h3>Practical Recommendations</h3><p>After running all three providers in production, here’s what we’ve learned:</p><h3>For Individual Users / Cost-Sensitive</h3><p><strong>Recommend: Apple on-device.</strong> Free, offline, private. The quality trade-off is minimal for typical reading and study use cases. This is our default for new users.</p><h3>For Quality-Focused Researchers</h3><p><strong>Recommend: OpenAI text-embedding-3-small.</strong> Higher dimensionality captures subtle semantic relationships better. Worth the cost for users processing technical papers where retrieval precision matters.</p><h3>For Users in China</h3><p><strong>Recommend: Doubao.</strong> Direct connection without VPN, reliable performance, competitive quality.</p><h3>The Hybrid Approach</h3><p>Some users start with Apple embeddings (free, instant) and later switch to OpenAI when they need higher retrieval quality for specific documents. The system handles this transition transparently — old caches are invalidated, new embeddings are generated, and retrieval quality improves without any data loss.</p><h3>Why User Choice Is a Competitive Advantage</h3><p>Most AI applications make the embedding choice for users. They either require cloud embeddings (locking out privacy-conscious users) or only offer on-device embeddings (limiting quality for users who want the best retrieval).</p><p>By supporting multiple providers with transparent trade-offs, we serve three audiences simultaneously:</p><ol><li><strong>Privacy-first users</strong> who would never use the app if it sent their documents to the cloud</li><li><strong>Quality-first users</strong> who want the best possible retrieval and are comfortable with cloud processing</li><li><strong>Cost-conscious users</strong> who want AI-powered reading without ongoing API expenses</li><li>This isn’t just a technical feature — it’s a product differentiator. In a market where most AI tools require cloud connectivity and ongoing subscriptions, offering a fully functional offline experience with optional cloud enhancement is rare.</li></ol><h3>What’s Next</h3><p>Over the past five articles, we’ve built a complete picture of production-grade AI engineering: from the failure modes of naive RAG (Part 1), through context assembly (Part 2), caching (Part 3), conversation memory (Part 4), and now embedding architecture (this article).</p><p>In <strong>Part 6</strong>, we’ll bring it all together with the launch of TT Notebook — the AI-powered reading assistant built on every system described in this series. We’ll show you how these components work in practice, from PDF reading to annotation to AI-powered discussion to automatic note generation.</p><p><em>I’m building </em><a href="https://apps.apple.com/cn/app/tt-notebook/id6758238388?mt=12"><em>TT Notebook</em></a><em>, an AI-powered reading assistant for macOS that implements everything in this series. Questions or ideas? Reach me at </em><a href="/service/mailto:support@tinytwo.me"><em>support@tinytwo.me</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=32c488be080f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Conversation Memory That Scales: Hierarchical Summarization for Long Chats]]></title>
            <link>https://medium.com/@guqung6/conversation-memory-that-scales-hierarchical-summarization-for-long-chats-1caa754e04fd?source=rss-e1958981e79b------2</link>
            <guid isPermaLink="false">https://medium.com/p/1caa754e04fd</guid>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[prompt-engineering]]></category>
            <category><![CDATA[ai-assistants-chatbots]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[conversational-ai]]></category>
            <dc:creator><![CDATA[TinyTwo]]></dc:creator>
            <pubDate>Mon, 11 May 2026 13:01:01 GMT</pubDate>
            <atom:updated>2026-05-11T13:01:01.793Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BbLoqbL5ZuHL4GoXoPbWiw.jpeg" /></figure><p><em>Part 4 of &quot;Building AI Agents That Actually Work&quot; — a series on production-grade AI engineering.</em></p><p>In <a href="https://medium.com/link-to-part-1">Part 1</a>, we identified conversation collapse as one of three failure modes that break RAG systems in production. Around message 40 or 50, the AI starts forgetting earlier context, repeating itself, and losing track of the conversation&#39;s original intent.</p><p>The root cause is arithmetic: conversation history grows linearly, but your token budget is fixed. At some point, the history alone exceeds what the context window can hold.</p><p>Most implementations solve this with a sliding window — keep the last N messages, drop everything else. This works for token management but creates an amnesia problem. The AI loses all memory of what happened before the window.</p><p>A step up is the <strong>single-summary + buffer</strong> pattern (LangChain&#39;s ConversationSummaryBufferMemory, LlamaIndex&#39;s SummaryMemoryBuffer): maintain one rolling global summary, keep the last N messages verbatim, and merge older messages into the summary when the budget tightens. This is meaningfully better than a sliding window, and it&#39;s the de-facto standard in production frameworks today.</p><p>In this article, I&#39;ll walk through a pragmatic extension of that pattern: <strong>two-tier hierarchical summarization</strong>. A single summary tends to degrade in quality across long conversations — everything past the buffer gets compressed into one blob, making it hard to tell &quot;the main topic from three days ago&quot; apart from &quot;the specific Q&amp;A two minutes ago.&quot; Splitting along the time axis (long-term / short-term / recent verbatim) gives different time ranges different fidelity, closer to how human memory actually behaves.</p><h3>The Problem with Sliding Windows</h3><p>Consider a 60-message conversation about a research paper. In messages 1–10, the user and AI discussed the paper&#39;s core thesis and identified three key arguments. In messages 20–30, they debated the methodology. In messages 50–60, the user asks: &quot;Going back to the first argument we discussed — how does the author&#39;s evidence for it compare to what we found in the methodology section?&quot;</p><p>With a 10-message sliding window, the AI has no memory of messages 1–10 or 20–30. It can&#39;t answer this question because it doesn&#39;t know what &quot;the first argument&quot; was or what was &quot;found in the methodology section.&quot;</p><p>You could increase the window size, but this just delays the problem. A 30-message window fails at message 60. A 50-message window consumes so many tokens that document context gets squeezed out — trading one failure mode for another.</p><p>The fundamental issue is that a sliding window treats all history as equally important. But in reality, the first messages (which establish context and goals) and the most recent messages (which contain the current thread) matter far more than messages in the middle.</p><h3>Hierarchical Summarization</h3><p>Our approach uses three tiers of memory, each with different compression levels:</p><pre>┌─────────────────────────────────────────┐<br>│ Long-Term Summary                       │<br>│ High compression: core topics, goals,   │<br>│ key conclusions                         │<br>│ Coverage: earliest messages             │<br>└─────────────────────────────────────────┘<br>                   ▼<br>┌─────────────────────────────────────────┐<br>│ Short-Term Summary                      │<br>│ Moderate compression: specific Q&amp;As,    │<br>│ key facts, ongoing discussions          │<br>│ Coverage: middle messages               │<br>└─────────────────────────────────────────┘<br>                   ▼<br>┌─────────────────────────────────────────┐<br>│ Recent Window (Last 10 Messages)        │<br>│ No compression: full verbatim content   │<br>│ Preserves tone, nuance, exact quotes    │<br>└─────────────────────────────────────────┘</pre><p>Each tier serves a different purpose:</p><ul><li><strong>Long-term summary</strong> preserves the &quot;why&quot; — what the conversation is about, what the user&#39;s goals are, what major decisions were made</li><li><strong>Short-term summary</strong> preserves the &quot;what&quot; — specific questions asked, facts discovered, topics explored</li><li><strong>Recent window</strong> preserves the &quot;how&quot; — exact wording, tone, ongoing reasoning chains</li><li>Together, they give the AI a complete picture: broad context from the beginning, moderate detail from the middle, and full fidelity for the current exchange.</li></ul><h3>When Summarization Triggers</h3><p>Summarization isn&#39;t free — it requires an LLM call that takes several seconds (the exact latency depends on which model you&#39;ve configured). Triggering it too often wastes time and money. Triggering it too late means the conversation has already degraded.</p><p>We use two trigger mechanisms:</p><h3>Trigger 1: Message Count Thresholds</h3><p>Total MessagesAction≤ 40No summarization — send full history41 (first trigger)Create short-term summary covering all messages except the last 10 (≈ 31 messages on first trigger)42–70Context = short-term summary + last 10 messages; the short-term summary <strong>rolls forward</strong> every ~20 new unsummarized messages~71+ (threshold)Promote short-term to long-term, create a new short-term summary (also requires &gt; 30 unsummarized messages)Steady stateContext = long-term + short-term + last 10 messages (both summaries continue to refresh as the conversation grows)</p><h3>Trigger 2: Token Pressure</h3><p>Message count alone doesn&#39;t catch all cases. Consider a conversation where a few messages include long quoted passages — 10 messages might consume as many tokens as 40 normal ones.</p><p>When conversation history tokens exceed <strong>80% of the history budget</strong>, summarization triggers immediately regardless of message count. This catches pathological cases where individual messages are unusually long.</p><p><strong>The history budget is a fixed 15% of the effective limit</strong> — roughly 22K tokens for Claude&#39;s 145,904 effective limit, ~14K for OpenAI&#39;s 128K, and ~3K for Doubao&#39;s 32K. We deliberately did <strong>not</strong> make it scale dynamically with document size. Dynamic scaling (bigger document → less room for history → earlier compression) sounds elegant in theory, but it creates inconsistent user experience: opening different documents would change how many turns the AI &quot;remembers.&quot; The real elasticity lives in other layers: documents are always compressed to Top-5 chunks via retrieval regardless of raw size, and notes/annotations are capped at 12%. Keeping the history budget at a fixed 15% makes conversation behavior predictable, testable, and debuggable.</p><h3>The Summarization Lifecycle</h3><p>Let&#39;s trace a conversation from start to finish:</p><h3>Messages 1–10: Full History</h3><p>All messages sent verbatim. No summarization needed.</p><pre>Context: [Message 1] [Message 2] ... [Message 10]<br>Token cost: ~4,000 tokens</pre><h3>Messages 11–40: Sliding Window</h3><p>History grows, but we still have budget. Messages 1–10 start to age but remain included.</p><pre>Context: [Message 1] [Message 2] ... [Message 40]<br>Token cost: ~16,000 tokens</pre><p>If the document is small, this might be fine — there&#39;s room in the budget. But for larger documents, the sliding window kicks in at 10+ messages:</p><pre>Context: [Message 31] [Message 32] ... [Message 40]<br>Token cost: ~4,000 tokens (10-message window)</pre><h3>Message 41: Short-Term Summary Created</h3><p>The 40-message threshold triggers. The system takes &quot;everything except the last 10 messages&quot; — on first trigger that&#39;s messages 1–31 (~31 total) — and generates a short-term summary:</p><p><strong>Prompt to LLM:</strong></p><blockquote><em>Summarize this conversation segment, preserving: specific questions asked and answered, key facts and data points discovered, important decisions or conclusions, topics still being explored, and any unresolved questions.</em></blockquote><p><strong>Output:</strong> A few hundred tokens of summary, with roughly 12:1 compression (the exact output length is determined by the LLM — we don&#39;t enforce a hard cap).</p><pre>Context: [Short-term summary] [Message 32] ... [Message 41]<br>Token cost: ~5,400 tokens (summary + 10 recent messages)</pre><h3>Messages 42–70: Rolling Updates + Window</h3><p>The conversation continues. Every ~20 new unsummarized messages, the short-term summary <strong>rolls forward</strong> to always cover &quot;all messages except the last 10.&quot; So the 42–70 range is not literally static — it just feels steady to the user because total token cost stays in a narrow band.</p><pre>Context: [Short-term summary (updated several times)] [Message 61] ... [Message 70]<br>Token cost: ~5,000 tokens</pre><h3>Threshold ~71+: Long-Term Promotion</h3><p>When message count exceeds 70 <strong>and</strong> there are more than 30 unsummarized messages, a two-step process triggers:</p><ol><li>The existing short-term summary is promoted to long-term (further compressed)</li><li>A new short-term summary is created for the next segment of middle messages</li><li><strong>Long-term summary prompt:</strong></li></ol><blockquote><em>Compress this summary further, retaining only: main topics and themes, overall user goals, key conclusions and decisions, and critical unresolved items.</em></blockquote><p><strong>Output:</strong> Compression ratio can reach ~40:1 (again, the LLM decides the actual length).</p><pre>Context: [Long-term summary] [Short-term summary] [Message 61] ... [Message 71]<br>Token cost: ~5,700 tokens</pre><h3>Rolling Steady State</h3><p>The pattern continues in rolling fashion: short-term updates every ~20 new messages, long-term promotions every additional ~30 unsummarized messages. The system always maintains:</p><ul><li>One long-term summary (highly compressed early context)</li><li>One short-term summary (moderately compressed middle context)</li><li>Last 10 messages in full</li><li>Token cost stays roughly constant, but both summaries themselves keep refreshing as the conversation evolves.</li></ul><h3>Integration with Context Assembly</h3><p>Hierarchical summarization doesn&#39;t exist in isolation — it&#39;s tightly coupled with the context assembly pipeline described in <a href="https://medium.com/link-to-part-2">Part 2</a>.</p><h3>Budget Interaction</h3><p>The history budget is <strong>a fixed 15% of the effective limit</strong> — it does not change with document size:</p><pre>Claude 200K   → 145,904 effective → ~22K history (100+ turns)<br>OpenAI 128K   → 91,904 effective  → ~14K history (60+ turns)<br>Doubao 32K    → 19,904 effective  → ~3K history (10–15 turns)</pre><p>Documents, annotations, and notes each absorb size pressure through their own mechanisms (documents via chunking + Top-5 retrieval; notes/annotations via a 12% cap), so the history budget doesn&#39;t need to be &quot;responsive.&quot; Keeping it fixed produces consistent conversation behavior across documents.</p><h3>Three Independent Layers, Not One Feedback Loop</h3><p>A natural question: &quot;Shouldn&#39;t a big document squeeze out history?&quot; In practice that intuition is already handled at a different layer:</p><ul><li><strong>Document layer</strong>: regardless of PDF size, only Top-5 chunks enter the prompt (typically a few K tokens). Documents don&#39;t balloon linearly.</li><li><strong>Annotation/Note layer</strong>: hard-capped at 12% of the budget.</li><li><strong>History layer</strong>: fixed 15% (~22K on Claude), enough for 100+ turns of typical conversation.</li><li>Each layer is independent, with no cross-coupling feedback loop. This makes the system easier to debug — if conversation behavior looks off, it&#39;s obvious which layer triggered compression, rather than having to untangle a chain of &quot;document size → history budget → summary threshold.&quot;</li></ul><h3>Assembly Order</h3><p>In the final prompt, summaries are placed before recent messages:</p><pre>[System prompt]<br>[Annotations and notes]<br>[Document content]<br>[Long-term summary: &quot;The conversation began with...&quot;]<br>[Short-term summary: &quot;Recent discussion covered...&quot;]<br>[Message 91: User asked about...]<br>[Message 92: Assistant responded...]<br>...<br>[Message 100: Current question]</pre><p>This ordering provides a natural narrative flow: broad context → recent context → current question. The LLM reads from general to specific, which helps it maintain coherence.</p><h3>What Summarization Preserves (and What It Loses)</h3><p>No compression is lossless. Understanding what gets preserved and what doesn&#39;t is critical for managing user expectations.</p><p><strong>Always preserved:</strong></p><ul><li>Core topics and themes of the conversation</li><li>User&#39;s stated goals and objectives</li><li>Major conclusions and decisions</li><li>Key facts and data points referenced</li><li>Unresolved questions and open threads</li><li><strong>Partially preserved:</strong></li><li>Specific questions (captured in short-term, lost in long-term)</li><li>Reasoning chains (compressed but traceable)</li><li>References to specific document pages or sections</li><li><strong>Lost:</strong></li><li>Exact wording and tone</li><li>Tangential discussions</li><li>Greetings and conversational filler</li><li>Repeated questions or clarifications</li><li>For most use cases, this trade-off is acceptable. The user asking &quot;What did we discuss earlier about optimization?&quot; will get an accurate answer from the summary. But &quot;What exact words did I use to describe my concern in message 5?&quot; will likely be lost once message 5 is summarized.</li></ul><h3>Persistence and Sync</h3><p>Summaries are stored as fields on the Conversation model in SwiftData:</p><ul><li>longTermSummary: text content</li><li>longTermSummarizedCount: number of messages covered</li><li>shortTermSummary: text content</li><li>shortTermSummarizedCount: number of messages covered</li><li>This means summaries persist across app restarts — no regeneration needed. For premium users, summaries sync across devices via CloudKit along with the conversation itself.</li><li>This is significant. Without persistence, reopening a 100-message conversation would require either regenerating summaries (expensive, slow) or sending all 100 messages to the LLM (likely exceeds the budget). With persistence, the app restores the same tiered context structure immediately.</li></ul><h3>Edge Cases and Safety Valves</h3><h3>The 80% Token Pressure Trigger</h3><p>The message-count triggers (40 and 70) work well for normal conversations. But what about a conversation where message 5 includes a 10,000-token document paste? Five messages could exceed the history budget.</p><p>The 80% pressure trigger catches this. If historyTokens &gt; 0.8 × historyBudget, summarization fires immediately, regardless of message count. This prevents conversations from silently degrading when individual messages are unusually large.</p><h3>Force Update</h3><p>Occasionally we need to trigger summarization outside of the standard thresholds — for example when the 80% token-pressure valve fires, or during debugging when we want to refresh a stale summary. A forceShortTermUpdate flag enables this without waiting for the message-count threshold.</p><h3>Empty History</h3><p>If a conversation has no messages yet (first question), summarization is skipped entirely. The system handles this gracefully by checking message count before attempting any history processing.</p><h3>Where This Sits on the Industry Spectrum</h3><p>To avoid overselling &quot;hierarchical summarization&quot; as novel, it helps to situate it next to what production systems actually do. Roughly four tiers of sophistication:</p><p><strong>Tier 1 · Sliding window.</strong> About 80% of stand-alone chatbots and API wrappers sit here. Everything past the window is dropped. Simple, amnesiac.</p><p><strong>Tier 2 · Single summary + buffer.</strong> The default pattern in LangChain, LlamaIndex, Continue.dev, AutoGPT, and most other production frameworks. One rolling global summary + last N messages. This is the de-facto industry standard today.</p><p><strong>Tier 3 · Tiered summarization + buffer (this article&#39;s approach).</strong> Adds one more layer on top of Tier 2. A long-term summary carries the earliest history (themes, goals, key conclusions), a short-term summary carries the middle (specific Q&amp;As), and the recent window carries full verbatim.</p><p><strong>Tier 4 · Virtual memory / semantic recall / fact extraction.</strong> Systems like MemGPT (Letta), Mem0, ReadAgent, and ChatGPT&#39;s &quot;Memory&quot; feature. The LLM actively decides what to page in and out, or retrieves past messages by semantic similarity, or extracts persistent user facts across conversations. An order of magnitude more engineering complexity than Tier 3.</p><p>TT Notebook sits at Tier 3 deliberately:</p><ul><li>For a bounded domain (&quot;conversation within one PDF&quot;), Tier 3 handles typical conversations up to ~100 turns well</li><li>Tier 4&#39;s MemGPT-style architecture adds huge value for agents and customer-support bots, but it&#39;s overkill for a reading assistant</li><li>Tier 2&#39;s single-summary approach starts showing quality decay past 50+ messages — both the early themes and recent specifics get muddled. Splitting the timeline mitigates that</li><li>Worth being honest: tiered summarization <strong>isn&#39;t novel</strong> — &quot;hierarchical memory&quot; is a classic taxonomy in the memory-systems literature. It&#39;s just that among shipping production apps, most stop at Tier 2.</li></ul><h3>What We Didn&#39;t Build</h3><p>To keep the positioning honest, here&#39;s what we deliberately left off:</p><p><strong>Cross-conversation memory.</strong> Each Conversation (typically one PDF + a thread) has its own independent summary. If you open two conversations about the same PDF on different days, the second won&#39;t inherit the first&#39;s summary. Mem0 and ChatGPT&#39;s Memory handle this, but they&#39;re extracting &quot;user facts&quot; (preferences, identity, goals) rather than summarizing conversations — two different memory paradigms with different sweet spots.</p><p><strong>Semantic recall over past messages.</strong> Suppose a user wrote a critical definition in message 5. Once it&#39;s folded into the short-term summary at message 80, that exact sentence may not survive compression. When the user later asks &quot;what was that render-pass definition you gave me?&quot;, we can only rely on whatever ended up in the summary. Embedding every past message and retrieving by semantic similarity would solve this, but it adds meaningful complexity to the context-assembly pipeline.</p><p><strong>AI-driven memory management.</strong> MemGPT&#39;s signature move is letting the LLM decide &quot;I need to look up that discussion from three weeks ago&quot; and call a recall_memory function itself. Our system is passive — summarization is triggered by message count or token pressure, not by the AI&#39;s own judgment. This matters for agents; for a reading assistant it&#39;s not the bottleneck.</p><p><strong>A dedicated summarization model.</strong> Summaries are generated by whatever LLM the user has configured for chat. A more mature approach would use a smaller, cheaper model (Haiku, GPT-4o-mini) specifically for summarization — saving main-model budget and reducing latency. We skipped it for two reasons: (1) it adds another API key for users to configure, and (2) model mismatch between &quot;summarizer&quot; and &quot;chat model&quot; can create subtle inconsistencies in what gets preserved. It&#39;s a sensible future optimization.</p><h3>Design Decisions</h3><p><strong>Why 10 messages for the recent window?</strong></p><p>Smaller windows (5 messages) caused the AI to lose track of multi-turn reasoning chains. Larger windows (20 messages) consumed too many tokens for users with tight budgets (e.g., Doubao&#39;s 32K context). Ten messages balances conversational coherence with token efficiency.</p><p><strong>Why 40 messages for the first summary trigger?</strong></p><p>Below 40 messages, the sliding window (10 messages) handles token management adequately. The summary adds value only when there&#39;s meaningful early context worth preserving — and 30+ messages of context (40 total minus 10 in the window) represents a substantial discussion worth summarizing.</p><p><strong>Why two summary tiers instead of one?</strong></p><p>A single summary would either be too detailed (defeating the purpose of compression) or too compressed (losing important specifics). Two tiers give us the best of both: the long-term summary captures the forest, the short-term summary captures the trees.</p><p><strong>Why not summarize incrementally (message by message)?</strong></p><p>Incremental summarization would add latency to every message. Batch summarization at thresholds means the cost is amortized — one multi-second LLM call covers ~30 messages. The user experiences one brief pause every 30+ messages instead of constant small delays.</p><p><strong>Why is the history budget a fixed 15% rather than dynamic?</strong></p><p>Dynamic allocation (bigger document → less history) sounds elegant in theory — &quot;adaptive,&quot; &quot;feedback loop,&quot; all the right words. In practice it makes user experience unpredictable: users can&#39;t understand why the AI remembers 50 prior turns with one PDF open but &quot;forgets&quot; everything when they switch to another. A fixed 15% decouples conversation behavior from document size, which makes it <strong>predictable, testable, and debuggable</strong>. The &quot;big documents squeezing history&quot; concern has already been solved one layer down, by chunking + retrieval — the document portion of the prompt is always Top-5 chunks, not proportional to total PDF size. Production-grade systems like Cursor, LangChain, and LlamaIndex overwhelmingly use fixed budgets for the same reason.</p><h3>What&#39;s Next</h3><p>We&#39;ve now covered the complete context assembly pipeline (Part 2), the caching architecture that makes it fast (Part 3), and the conversation memory system that makes it scale (this article). In <strong>Part 5</strong>, we&#39;ll tackle a different dimension of the problem: embeddings. Specifically, the trade-offs between on-device and cloud embeddings, and why giving users a choice between privacy and quality is a competitive advantage.</p><p><em>I&#39;m building </em><a href="https://apps.apple.com/cn/app/tt-notebook/id6758238388?mt=12"><em>TT Notebook</em></a><em>, an AI-powered reading assistant for macOS that implements everything in this series. Questions or ideas? Reach me at </em><a href="/service/mailto:support@tinytwo.me"><em>support@tinytwo.me</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1caa754e04fd" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Smart Caching for LLM Apps: Three Tiers from Memory to API]]></title>
            <link>https://medium.com/@guqung6/smart-caching-for-llm-apps-three-tiers-from-memory-to-api-47ee53a68998?source=rss-e1958981e79b------2</link>
            <guid isPermaLink="false">https://medium.com/p/47ee53a68998</guid>
            <category><![CDATA[ai-engineering]]></category>
            <category><![CDATA[caching]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[design-systems]]></category>
            <category><![CDATA[llm]]></category>
            <dc:creator><![CDATA[TinyTwo]]></dc:creator>
            <pubDate>Mon, 04 May 2026 13:01:01 GMT</pubDate>
            <atom:updated>2026-05-04T13:01:01.925Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ambiYUWMm2WRCR7xNVaNcQ.jpeg" /></figure><p><em>Part 3 of “Building AI Agents That Actually Work” — a series on production-grade AI engineering.</em></p><p>In <a href="https://medium.com/link-to-part-2">Part 2</a>, we built a context assembly pipeline with 8 steps: text extraction, structural classification, annotation retrieval, note retrieval, budget allocation, document processing, history management, and final assembly. On paper, this pipeline produces excellent prompts. In practice, without caching, it’s unusably slow.</p><p>Here’s what happens without caching: every question requires re-extracting the full PDF text (1–5 seconds), regenerating vector embeddings (5–60 seconds), re-parsing the document outline (up to 15 seconds with LLM fallback), and re-computing relevance scores. The user waits 30+ seconds for each answer.</p><p>With caching, the same pipeline runs in 50–200 milliseconds for cached documents. That’s not a rounding error — it’s the difference between a usable product and an abandoned one.</p><p>In this article, I’ll walk through the three-tier caching architecture that makes context assembly fast and affordable: memory caches for millisecond access, disk caches for cross-session persistence, and API-level caches for 90% cost reduction.</p><h3>The Three Tiers</h3><pre>┌─────────────────────────────────────────────┐<br>│ Tier 3: API Cache (Server-Side)             │<br>│ Prompt Cache — managed by LLM provider      │<br>│ TTL ~5 min, cached tokens cost 10% of normal│<br>└─────────────────────────────────────────────┘<br>                    ▲<br>┌─────────────────────────────────────────────┐<br>│ Tier 2: Disk Cache (Cross-Session)          │<br>│ Embedding JSON files · Outline JSON files   │<br>│ Persists across app restarts                │<br>└─────────────────────────────────────────────┘<br>                    ▲<br>┌─────────────────────────────────────────────┐<br>│ Tier 1: Memory Cache (Millisecond Access)   │<br>│ Document text · Embeddings · Outlines       │<br>│ Core ML model instances · User settings     │<br>└─────────────────────────────────────────────┘</pre><p>Each tier serves a different purpose. Memory caches eliminate redundant computation within a session. Disk caches survive app restarts and prevent expensive regeneration. API caches reduce the cost of repeated LLM calls.</p><h3>Tier 1: Memory Caches</h3><h3>Document Text Cache</h3><p>Every AI question needs the document’s full text. Extracting text from a PDF via PDFKit takes 1–5 seconds depending on document size. Without caching, five questions on the same document means five extractions.</p><p><strong>Design:</strong></p><ul><li><strong>Storage:</strong> In-memory dictionary keyed by document UUID</li><li><strong>TTL:</strong> 1 hour (3600 seconds)</li><li><strong>Validation:</strong> Dual check on every access — cache age AND page count match</li><li><strong>No disk persistence:</strong> Deliberately omitted. PDF text extraction is fast enough that re-extracting after an app restart is acceptable. Adding disk I/O and file management complexity isn’t worth it.</li><li><strong>Token estimation</strong> is embedded in this cache. When text is extracted, we estimate its token count using language-aware heuristics: ~4 characters per token for Latin text, ~1.5 characters per token for CJK text. This estimate feeds directly into the token budget allocator in Step 5 of the pipeline.</li><li>Why not just tokenize exactly? Real tokenizers are provider-specific and relatively slow. An approximate count with a 25% safety buffer (applied later in budgeting) is faster and good enough.</li></ul><h3>Embedding Memory Layer</h3><p>Vector embeddings have a two-tier access pattern: memory first, disk second. The memory layer is a simple dictionary that provides sub-millisecond lookups.</p><p>When embeddings are requested:</p><ol><li>Check memory dictionary → hit returns immediately</li><li>Miss → check disk JSON file → hit loads into memory, then returns</li><li>Miss → return empty (caller generates new embeddings)</li><li>Write path: embeddings are written to memory immediately, then flushed to disk asynchronously with atomic writes.</li></ol><h3>Core ML Model Instance Cache</h3><p>On-device embedding uses a Core ML model (~50MB). Loading it takes several seconds. The model instance is cached as a singleton for the app’s lifetime — loaded lazily on first use, with a warmUp() method for optional preloading at startup.</p><p>A loading guard prevents concurrent initialization: if two requests arrive while the model is loading, the second waits for the first to complete rather than triggering a duplicate load.</p><h3>Tier 2: Disk Caches</h3><h3>Embedding Disk Cache</h3><p>This is the most complex cache in the system. It manages three separate sub-caches, each stored as JSON files per document:</p><p>Sub-CacheContentsDirectoryPurposeDocument chunk embeddingsVector arrays for text chunksembeddings/Semantic retrieval of relevant passagesAnnotation embeddingsVectors for highlights + user notesannotation-embeddings/Semantic retrieval of relevant annotationsNote embeddingsVectors for study notesnote-embeddings/Semantic retrieval of relevant notes</p><p>All files live under ~/Library/Application Support/AIReadingAssistant/.</p><p><strong>Three-Dimensional Validation</strong></p><p>On every disk cache read, three checks run. Any failure invalidates the entire cache for that document:</p><ol><li><strong>Page count</strong> (document embeddings only) — if the PDF gained or lost pages, the chunks are stale</li><li><strong>Model name</strong> — different models produce vectors in different semantic spaces; mixing them corrupts retrieval</li><li><strong>Provider</strong> — even with the same model name, different providers (Apple / OpenAI / Doubao) may produce different embeddings</li><li>Why three dimensions? We learned this the hard way. A user switched from OpenAI to Apple embeddings, and the cached OpenAI vectors were being compared against new Apple query vectors. Cosine similarity between vectors from different embedding spaces is meaningless — it returns near-random results. The three-dimensional check prevents this silently.</li><li><strong>Incremental Updates</strong></li><li>For annotations and notes, full regeneration is wasteful. If a user adds one annotation to a document with 50 existing annotations, regenerating all 50 embeddings costs time and money.</li><li>Instead, the system compares updatedAt timestamps:</li></ol><ul><li>If annotation.updatedAt ≤ cachedEmbedding.timestamp → reuse existing embedding</li><li>If annotation.updatedAt &gt; cachedEmbedding.timestamp → regenerate only that embedding</li><li>New annotations → generate and append</li><li>This means adding or editing a single annotation triggers generation of exactly one embedding, not fifty.</li></ul><h3>Outline Disk Cache</h3><p>Document outlines (table of contents) are cached as JSON files under outlines/. The outline retrieval system uses a five-level fallback chain:</p><pre>1. Memory cache → hit → return (milliseconds)<br>2. Disk cache → hit → load to memory, return (~50ms)<br>3. Native PDF bookmarks (PDFKit) → found → cache and return (&lt;100ms)<br>4. LLM extraction (scan first 10 pages for TOC) → success → cache and return (5-15s)<br>5. LLM full-text scan (small documents only, &lt;50 pages) → cache and return</pre><p><strong>Smart empty handling:</strong> The cache tracks whether an outline came from native PDF bookmarks or LLM extraction. If native parsing found nothing (the PDF has no bookmarks), but the AI service becomes available later, the system retries LLM extraction. But if LLM already tried and found nothing, it won’t retry — avoiding wasted API calls.</p><h3>Tier 3: API-Level Cache (Prompt Cache)</h3><p>This tier operates on the LLM provider’s servers, not locally. It caches the system prompt — which includes instructions, document content, and annotations — so that repeated questions about the same document don’t re-process the same tokens.</p><h3>How It Works</h3><pre>Request 1 (cache creation):<br>  [system prompt: instructions + document + annotations]  ← full price, marked cacheable<br>  [conversation history]<br>  [current question]<br>  → Provider caches the system prompt portion<br>​<br>Request 2 (within 5 minutes, cache hit):<br>  [system prompt: identical to Request 1]  ← 10% price (cache hit)<br>  [conversation history: +1 exchange]<br>  [current question: new question]<br>  → Provider uses cached system prompt</pre><h3>Provider Support</h3><p>ProviderMechanismMarkingTTLClaudePrompt CacheExplicit cache_control: ephemeral~5 min (refreshed on hit)OpenAIAutomatic prefix matchingNo marking neededServer-managedDoubaoNone——</p><h3>Prompt Ordering for Cache Efficiency</h3><p>The order of blocks in the assembled prompt directly affects cache hit rates. We structure prompts with static content first and dynamic content last:</p><ol><li><strong>System instructions</strong> — identical across all requests (perfect for caching)</li><li><strong>Notes and annotations</strong> — stable within a document session</li><li><strong>Document content</strong> — stable within a document session</li><li><strong>Conversation history</strong> — changes every turn</li><li><strong>Current question</strong> — changes every turn</li><li>Because prompt caching works on prefix matching, everything before the first change is cached. By putting stable content at the beginning, we maximize the cached prefix length.</li></ol><h3>Cost Impact</h3><p>In a typical reading session where a user asks 5 questions about a document:</p><p>QuestionSystem Prompt TokensCache StatusEffective CostQ1~15,000Cache creation100%Q2~15,000Cache hit10%Q3~15,000Cache hit10%Q4~15,000Cache hit10%Q5~15,000Cache hit10%</p><p>That’s a 72% reduction in system prompt token costs across the session. For Claude, which supports explicit cache control, the savings are predictable and reliable.</p><h3>How the Tiers Work Together</h3><p>Here’s what happens when a user opens a document and asks their first question:</p><p><strong>First question (cold start):</strong></p><ol><li>Text extraction: 1–5s (no cache) → cached in memory</li><li>Outline parsing: &lt;100ms (native) or 5–15s (LLM) → cached in memory + disk</li><li>Embedding generation: 3s (Apple) or 10–30s (OpenAI) → cached in memory + disk</li><li>Context assembly: runs pipeline → prompt assembled</li><li>LLM call: system prompt cached on provider’s server (Tier 3)</li><li><strong>Second question (warm):</strong></li><li>Text extraction: &lt;1ms (memory hit)</li><li>Outline: &lt;1ms (memory hit)</li><li>Embeddings: &lt;1ms (memory hit)</li><li>Context assembly: 50–200ms total</li><li>LLM call: system prompt 90% cheaper (Tier 3 cache hit)</li><li><strong>After app restart, same document:</strong></li><li>Text extraction: 1–5s (memory miss, re-extract) → re-cached</li><li>Outline: ~50ms (disk hit) → loaded to memory</li><li>Embeddings: ~100ms (disk hit) → loaded to memory</li><li>Context assembly: runs normally</li><li>LLM call: likely cache miss (Tier 3 TTL expired during restart)</li><li><strong>Switching embedding provider:</strong></li><li>All embedding caches invalidated (provider dimension fails validation)</li><li>Full regeneration: 5–60s depending on provider</li><li>Text cache and outline cache unaffected</li></ol><h3>Performance Characteristics</h3><h3>Latency</h3><p>OperationMemory HitDisk HitCache MissDocument text&lt;1ms — 1–5sEmbeddings&lt;1ms~100ms5–60sOutline&lt;1ms~50ms&lt;100ms or 5–15sVector search (5 chunks)~10ms — Requires cached embeddingsFull pipeline50–200ms — 10–60s</p><h3>Memory Footprint</h3><p>ComponentPer Document20 DocumentsDocument text10KB–5MB200KB–100MBChunk embeddings100KB–5MB2MB–100MBOutline1–50KB20KB–1MBCore ML model~50MB (global singleton)~50MB</p><h3>Disk Footprint</h3><p>ComponentPer Document20 DocumentsEmbedding JSON (chunks)100KB–10MB2MB–200MBEmbedding JSON (annotations)1KB–1MBvariableEmbedding JSON (notes)1KB–1MBvariableOutline JSON1–50KB20KB–1MB<strong>Total</strong> — <strong>~50–200MB</strong></p><h3>Cache Invalidation Rules</h3><p>Getting invalidation right is the hardest part of caching. Here’s the complete invalidation matrix:</p><h3>Automatic Invalidation</h3><p>EventAffected CachesMechanismPDF page count changesDocument text + chunk embeddingsValidation check on accessEmbedding model changedAll embeddingsModel dimension mismatch on accessEmbedding provider changedAll embeddingsProvider field mismatch on access1 hour elapsedDocument text onlyTTL expirationApp restartAll memory cachesProcess termination</p><h3>Manual Invalidation</h3><p>ActionScopeDelete documentAll caches for that documentClear embedding cache (Settings)All embedding caches (memory + disk)Switch embedding modelAll embeddings (fails validation on next access)Switch embedding providerAll embeddings (fails validation on next access)</p><p>The key principle: <strong>caches are validated on read, not invalidated on write.</strong> When a user switches their embedding provider in settings, we don’t immediately delete all cached embeddings. Instead, the next time embeddings are requested, the validation check detects the provider mismatch and triggers regeneration. This is simpler, safer, and handles edge cases naturally.</p><h3>Design Decisions</h3><p><strong>Why not use a general-purpose cache library?</strong></p><p>Each cache has unique validation requirements. Document text validates on age + page count. Embeddings validate on model + provider + page count. Outlines validate on page count and track their source. A generic cache with TTL-only invalidation would miss these domain-specific checks.</p><p><strong>Why JSON for disk storage instead of SQLite?</strong></p><p>Embeddings are large, write-once, read-many data structures. JSON files are simple to implement, easy to debug (you can open them in a text editor), and atomic writes prevent corruption. SQLite would add complexity without meaningful benefits for this access pattern.</p><p><strong>Why no LRU eviction?</strong></p><p>Currently, the disk cache has no size limit. For a typical user with 20 documents, disk usage is 50–200MB — manageable. Adding LRU eviction is on the roadmap for users with larger libraries, but premature optimization wasn’t worth the complexity for v1.</p><p><strong>Why 1-hour TTL for document text?</strong></p><p>The cache exists to prevent redundant extraction during a reading session, not for long-term storage. One hour covers the typical session length while ensuring that externally modified PDFs are eventually re-extracted. Text extraction is fast enough (1–5s) that a miss isn’t painful.</p><h3>What’s Next</h3><p>Caching makes the pipeline fast and affordable, but there’s one component we’ve only touched on: conversation memory. In <strong>Part 4</strong>, we’ll dive deep into hierarchical summarization — how to compress 100-message conversations into a tiered memory structure that preserves both broad context and recent detail, enabling unlimited conversation length within a fixed token budget.</p><p><em>I’m building </em><a href="https://apps.apple.com/cn/app/tt-notebook/id6758238388?mt=12"><em>TT Notebook</em></a><em>, an AI-powered reading assistant for macOS that implements everything in this series. Questions or ideas? Reach me at </em><a href="/service/mailto:support@tinytwo.me"><em>support@tinytwo.me</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=47ee53a68998" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Context Assembly: The Missing Layer Between Your LLM and Your Data]]></title>
            <link>https://medium.com/@guqung6/context-assembly-the-missing-layer-between-your-llm-and-your-data-c62220e6b542?source=rss-e1958981e79b------2</link>
            <guid isPermaLink="false">https://medium.com/p/c62220e6b542</guid>
            <category><![CDATA[software-architecture]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[rags]]></category>
            <category><![CDATA[prompt-engineering]]></category>
            <category><![CDATA[llm]]></category>
            <dc:creator><![CDATA[TinyTwo]]></dc:creator>
            <pubDate>Mon, 27 Apr 2026 13:06:01 GMT</pubDate>
            <atom:updated>2026-04-27T13:06:01.363Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Gn-Sqb0OgZrW00Cr4fqyrQ.jpeg" /></figure><p><em>Part 2 of “Building AI Agents That Actually Work” — a series on production-grade AI engineering.</em></p><p>In <a href="https://medium.com/link-to-part-1">Part 1</a>, we identified three failure modes that break RAG systems in production: token explosion, imprecise retrieval, and conversation collapse. They aren’t independent problems — they interact, compound, and create a downward spiral of quality degradation.</p><p>The solution isn’t fixing each problem in isolation. It’s building an orchestration layer that manages all three simultaneously. I call this layer <strong>context assembly</strong> — and it’s the most important architectural component your LLM application is probably missing.</p><p>In this article, I’ll walk through the complete 8-step context assembly pipeline we built for a production reading assistant, including the exact scoring formulas, token budgets, and decision trees that make it work.</p><h3>Why “Just Use RAG” Isn’t Enough</h3><p>Traditional RAG has one job: retrieve relevant chunks and send them to the LLM. Context assembly has seven:</p><ol><li><strong>Budget allocation</strong> — How many tokens can each component use?</li><li><strong>Structural understanding</strong> — Is the user asking about a specific section?</li><li><strong>Multi-source retrieval</strong> — Annotations, notes, and document chunks are different things</li><li><strong>Strategy selection</strong> — Full document, truncated, or chunked? It depends on the budget</li><li><strong>History management</strong> — Full history, sliding window, or summarized?</li><li><strong>Relevance scoring</strong> — Combine vector similarity with position, structure, and content type</li><li><strong>Assembly ordering</strong> — The order of context blocks affects both quality and cost</li><li>A retrieval pipeline handles #6. Context assembly handles all seven.</li></ol><h3>The 8-Step Pipeline</h3><p>Here’s the complete flow, from user question to assembled prompt:</p><pre>User Question<br>     │<br>     ▼<br>┌──────────────────────────────────────┐<br>│ Step 1: Document Text Cache          │<br>│ Step 2: Structural Query Classification │<br>│ Step 3: Annotation Retrieval         │<br>│ Step 4: Note Retrieval               │<br>│ Step 5: Token Budget Allocation      │<br>│ Step 6: Document Content Processing  │<br>│ Step 7: Conversation History         │<br>│ Step 8: Final Assembly               │<br>└──────────────────────────────────────┘<br>     │<br>     ▼<br>  LLM Prompt</pre><p>Each step feeds into the next. Let’s walk through them.</p><h3>Step 1: Document Text Extraction &amp; Caching</h3><p>Before anything else, we need the document’s full text. PDF text extraction takes 1–3 seconds for a typical document — acceptable for a single question, but unacceptable when the user asks five questions in a row.</p><p>We cache extracted text in memory with a 1-hour TTL. The cache key includes the document ID and page count, so if the PDF changes (pages added or removed), the cache automatically invalidates.</p><p>Token estimation happens here too, and it’s trickier than you’d think. English text runs roughly 4 characters per token. Chinese text runs 1.5 characters per token. A bilingual system needs to handle both, or its budget calculations will be wildly off.</p><h3>Step 2: Structural Query Classification</h3><p>Not all questions are equal. “What is gradient descent?” is a semantic question — vector similarity handles it well. But “What does Chapter 5 say about optimization?” is a structural question — the user is pointing at a specific section of the document.</p><p>We use a lightweight LLM call to classify the query against the document’s table of contents:</p><p><strong>Input:</strong> Document outline + user question <strong>Output:</strong></p><pre>{<br>  &quot;useStructure&quot;: true,<br>  &quot;targetChapters&quot;: [&quot;Chapter 5: Optimization Methods&quot;],<br>  &quot;includeOutlineInContext&quot;: false,<br>  &quot;reasoning&quot;: &quot;User explicitly references Chapter 5&quot;<br>}</pre><p>The classifier returns target chapters, which later steps use to boost retrieval relevance. Chapter matching uses a three-tier fallback:</p><ol><li><strong>Exact match</strong> — The LLM’s output exactly matches an outline entry</li><li><strong>Contains match</strong> — One string is a substring of the other</li><li><strong>Fuzzy match</strong> — Two or more shared words between the title and outline entry</li><li>Why three tiers? Because LLMs don’t always reproduce chapter titles exactly. They might say “Chapter 5” when the outline says “Chapter 5: Optimization Methods,” or paraphrase the title entirely. Fuzzy matching catches these cases without being so loose that it matches everything.</li></ol><h3>Step 3: Annotation Retrieval</h3><p>Annotations are special. Unlike document chunks, they represent content the user explicitly decided was important. They carry intent signals that raw text doesn’t.</p><p><strong>With embeddings available</strong>, we score each annotation using a composite formula:</p><pre>score = cosineSimilarity(query, annotation)<br>      + pageBoost<br>      + noteBoost</pre><p>Where:</p><ul><li><strong>pageBoost</strong>: +0.15 for annotations on the current page, +0.05 for adjacent 2 pages</li><li><strong>noteBoost</strong>: +0.03 if the annotation has a user-written note attached</li><li>We select the top 8 annotations by this combined score. Any annotation the user explicitly referenced (e.g., by selecting it in the UI) bypasses scoring entirely and is always included.</li><li><strong>Without embeddings</strong>, the system falls back to proximity-based selection — take the nearest annotations to the current page. It works, but it misses cross-document connections.</li><li>Why cap at 8? Annotations are dense with meaning (the user highlighted them for a reason), so each one contributes more information per token than a random document chunk. But they can also be misleading if too many unrelated annotations dilute the context. Eight is the empirically-determined sweet spot.</li></ul><h3>Step 4: Note Retrieval</h3><p>Independent notes — user-written study notes, AI-generated summaries — are a distinct content type. They’re longer than annotations, more structured, and sometimes contain the user’s own analysis.</p><p>The retrieval algorithm mirrors Step 3 but with different boosting:</p><pre>score = cosineSimilarity(query, note)<br>      + annotationPageBoost<br>      + priorityBoost</pre><p>Where:</p><ul><li><strong>annotationPageBoost</strong>: +0.1 if the note’s associated annotations appear on the current page</li><li><strong>priorityBoost</strong>: priority level × 0.03</li><li>We take the top 5 notes. Before embedding, note content goes through cleaning: Base64-encoded images are stripped (they create noise in embeddings), and HTML/Markdown formatting is removed.</li></ul><h3>Step 5: Token Budget Allocation</h3><p>This is where the pipeline gets interesting. We need to divide a finite token budget across competing needs, with hard guarantees for each.</p><p><strong>The budget framework:</strong></p><pre>effectiveLimit = maxContextTokens × 0.75 - 4,096<br>​<br>Fixed costs (non-negotiable):<br>├─ System prompt: ~500 tokens<br>├─ Current question: variable<br>└─ Notes + annotations: min(actual, 12% of effectiveLimit)<br>​<br>Flexible costs (adaptive):<br>├─ Document: remaining - history (min 10,000 guaranteed)<br>└─ History: based on strategy selection</pre><p>The 25% safety buffer is critical. Token estimation is inherently imprecise — different tokenizers produce different counts, and CJK text is especially hard to estimate. Without this buffer, edge cases will blow past the context window and cause hard failures.</p><p><strong>Document strategy selection</strong> is a decision tree:</p><pre>Can full document fit in budget?<br>├─ Yes → Send full document<br>└─ No → Is budget ≥ 10,000 tokens?<br>         ├─ Yes → Truncate to budget<br>         └─ No → Use chunked retrieval</pre><p><strong>History strategy selection</strong> runs in parallel:</p><pre>More than 40 messages AND no summary exists?<br>├─ Yes → Request summarization<br>└─ No → More than 10 messages?<br>         ├─ Yes → Sliding window (keep last 10)<br>         └─ No → Send full history</pre><p>The key insight is that these strategies interact. If the document is large, history gets a smaller budget, pushing it toward summarization earlier. If the conversation is long, the document budget shrinks, pushing it toward chunked retrieval. The budget manager handles these trade-offs automatically.</p><h3>Step 6: Document Content Processing</h3><p>This step has three paths, chosen in priority order:</p><p><strong>Path A — Embedding-Based Retrieval (optimal)</strong></p><p>When embeddings are available, the system:</p><ol><li>Chunks the document using adaptive sizing:</li></ol><p>Document SizeChunk SizeOverlap&lt; 5,000 tokensSingle chunkNone5K–50K tokens3,000 tokens300 tokens&gt; 50K tokens2,000 tokens200 tokens</p><p>2. Generates a query embedding</p><p>3. Scores each chunk: similarity + pageBoost(0.1) + structuralBoost(0.3)</p><ol><li>Selects the top 5 chunks</li><li>Chunk boundaries are placed intelligently: paragraph breaks first (\n\n), then sentence boundaries (.!?), then character boundaries as a last resort. The overlap ensures that content spanning chunk boundaries isn&#39;t lost.</li><li>Why do larger documents get smaller chunks? Precision. A 200-page textbook has many potential answers to any question. Smaller chunks mean the retrieval system can be more selective, returning only the most relevant passages instead of large blocks of mixed-relevance content.</li><li>When structural classification identifies target chapters, retrieval uses priority filling: chunks from target chapters fill slots first (sorted by similarity), then remaining slots are filled with the highest-similarity chunks from anywhere in the document. This ensures chapter-specific questions get chapter-specific answers, without completely ignoring relevant content elsewhere.</li><li><strong>Path B — Structural Extraction (fallback)</strong></li><li>When the query targets specific chapters but embeddings aren’t available, the system extracts text directly from the identified page ranges. It’s less precise than embedding-based retrieval but far better than sending the full document.</li><li><strong>Path C — Direct Inclusion (safety net)</strong></li><li>For small documents or when neither embeddings nor structural information is available:</li></ol><ul><li>Documents under budget → send in full</li><li>Documents over budget but under 50K → truncate</li><li>Documents over 50K → forced chunking (truncate to 50K equivalent)</li><li>The 50,000-token forcing threshold exists because sending a document that large without any selection is almost certainly wasteful. Even without embeddings, truncation is better than nothing.</li></ul><h3>Step 7: Conversation History Processing</h3><p>By now, the budget for history is determined. This step formats it:</p><ul><li><strong>Full history</strong> (≤10 messages): Include everything verbatim</li><li><strong>Sliding window</strong> (11–39 messages): Prepend any existing tiered summaries, then include the last 10 messages</li><li><strong>Summarized</strong> (40+ messages): Create or update summaries, then use tiered format</li><li>The tiered summary structure:</li></ul><pre>[Long-term summary: &quot;Major themes discussed...&quot;]<br>[Short-term summary: &quot;Recent questions about...&quot;]<br>[Message 91] User: &quot;...&quot;<br>[Message 92] Assistant: &quot;...&quot;<br>... (last 10 messages in full)</pre><p>There’s also a token-pressure escape valve: if history tokens exceed 80% of the history budget at any point, summarization triggers immediately, regardless of message count. This catches pathological cases where a few very long messages eat the entire budget.</p><h3>Step 8: Final Assembly</h3><p>The last step combines everything into a single prompt, in a specific order:</p><ol><li><strong>System prompt</strong> — Fixed instruction set (~500 tokens)</li><li><strong>Notes and annotations</strong> — User’s annotations and study notes</li><li><strong>Document content</strong> — Retrieved chunks, structural extracts, or full document</li><li><strong>Conversation history</strong> — Summaries + recent messages</li><li><strong>Current question</strong> — The user’s latest input</li><li>This ordering isn’t arbitrary. It’s optimized for prompt caching. The system prompt is identical across requests and sits at the beginning, maximizing cache hit rates with providers that support cache_control (like Claude’s Prompt Cache). We’ll cover this in detail in Part 3.</li></ol><h3>How the Components Interact</h3><p>The beauty of context assembly is that each component adapts to the others:</p><pre>Large document (80K tokens)<br>  → Document takes most of budget<br>  → History squeezed → Summarization triggered earlier<br>  → Fewer annotation slots available → Only top-scoring survive</pre><pre>Small document (5K tokens)<br>  → Document uses minimal budget<br>  → History gets generous allocation → Full history preserved longer<br>  → More room for annotations and notes</pre><pre>Long conversation (60 messages)<br>  → History needs significant tokens even after summarization<br>  → Document budget reduced → Chunked retrieval activated<br>  → Annotations more critical (compensate for less document context)</pre><p>These interactions happen automatically through the budget allocation algorithm. No special-case code needed — the same pipeline handles a 5-page paper with 3 questions and a 300-page textbook with 100-message conversation.</p><h3>Real Numbers</h3><p>Here’s a token flow example for a typical session on Claude (200K context):</p><p>ComponentTokensStrategyEffective limit145,904200K × 0.75–4,096System prompt500FixedQuestion200VariableAnnotations (6)1,800Embedding retrievalNotes (2)1,200Priority-basedDocument (5 chunks)15,000Embedding retrievalHistory (25 msgs)8,000Sliding window<strong>Total used26,700Remaining buffer119,204</strong>Safety margin</p><p>The system deliberately under-fills the context window. That remaining buffer handles response generation and provides safety margin for token estimation errors.</p><h3>Key Design Decisions</h3><p><strong>Why 12% cap for notes/annotations?</strong> Without a cap, a document with 50 annotations could consume the entire budget before document content is even considered. 12% ensures annotations inform the AI without dominating its context.</p><p><strong>Why top 5 chunks instead of more?</strong> Retrieval has diminishing returns. The 5th most relevant chunk adds less value than the 1st. Beyond 5, you’re trading document breadth for retrieval noise. This number was tuned empirically.</p><p><strong>Why 25% safety buffer (not 10% or 30%)?</strong> We started at 15% and hit context overflow errors with CJK-heavy documents. 25% eliminates these failures while leaving enough room for useful content. 30% was too aggressive — it wasted too much capacity.</p><p><strong>Why classify queries before retrieval?</strong> Without structural classification, “What’s in Chapter 5?” would return the top-5 semantically similar chunks, which might be spread across the entire document. The +0.3 structural boost ensures chapter-specific questions get chapter-specific answers.</p><h3>What’s Next</h3><p>Context assembly solves the orchestration problem, but it depends on fast access to document text, embeddings, and conversation summaries. In <strong>Part 3</strong>, we’ll cover the three-tier caching architecture that makes this pipeline run in milliseconds instead of seconds — and cuts API costs by 90%.</p><p><em>I’m building </em><a href="https://apps.apple.com/cn/app/tt-notebook/id6758238388?mt=12"><em>TT Notebook</em></a><em>, an AI-powered reading assistant for macOS that implements everything in this series. Questions or ideas? Reach me at </em><a href="/service/mailto:support@tinytwo.me"><em>support@tinytwo.me</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c62220e6b542" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why Most RAG Implementations Fail — and What to Do Instead]]></title>
            <link>https://medium.com/@guqung6/why-most-rag-implementations-fail-and-what-to-do-instead-36fa8b7af00a?source=rss-e1958981e79b------2</link>
            <guid isPermaLink="false">https://medium.com/p/36fa8b7af00a</guid>
            <category><![CDATA[rags]]></category>
            <category><![CDATA[retrieval-augmented]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[claude-code]]></category>
            <category><![CDATA[ai-engineering]]></category>
            <dc:creator><![CDATA[TinyTwo]]></dc:creator>
            <pubDate>Mon, 20 Apr 2026 13:01:01 GMT</pubDate>
            <atom:updated>2026-04-20T13:01:01.535Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GXH90TDrttQcOLUvMuZ4BQ.jpeg" /></figure><p><em>This is Part 1 of “Building AI Agents That Actually Work,” a series on production-grade AI engineering.</em></p><p>You’ve built a RAG system. You chunked your documents, generated embeddings, wired up a vector database, and connected it to an LLM. It works — until it doesn’t.</p><p>The demo looked great. But in production, your users start reporting problems: the AI ignores relevant information, contradicts itself mid-conversation, and occasionally hallucinates answers it could have found in the source material. Sound familiar?</p><p>After building an AI-powered reading assistant that handles everything from 5-page papers to 300-page textbooks, I’ve learned that the gap between a RAG demo and a production RAG system is enormous. In this article, I’ll walk through the three failure modes that plague most RAG implementations and introduce the architectural patterns that solve them.</p><h3>Failure Mode #1: Token Explosion</h3><p>The most common RAG approach is simple: take the user’s question, retrieve relevant chunks, stuff them into a prompt alongside the question, and send everything to the LLM. This works fine when your document fits within the context window.</p><p>But what happens when it doesn’t?</p><p>Consider a real scenario. A user opens a 100-page PDF — roughly 50,000 tokens. They ask a question, and the system retrieves relevant chunks. But there’s also conversation history from previous questions, annotations the user has made, and study notes they’ve written. Add the system prompt and response reservation, and suddenly you’re looking at 70,000+ tokens competing for space.</p><p>Here’s where the math gets uncomfortable:</p><p>ProviderRaw Context WindowAfter Safety Buffer (25%)Response ReservationEffective BudgetClaude200K150K4,096~146KOpenAI128K96K4,096~92KDoubao32K24K4,096~20K</p><p>That 25% safety buffer isn’t paranoia — it accounts for token estimation errors between different languages and encoding schemes. Without it, your system will silently truncate context at the worst possible moment.</p><p>The real problem isn’t just fitting content into the window. It’s that naive RAG treats all content as equally important. A chunk from the relevant chapter competes for tokens against a chunk from the appendix. An insightful user annotation gets the same priority as a routine page header. Without deliberate prioritization, your most valuable context gets displaced by noise.</p><p><strong>The fix: Token budgeting.</strong> Instead of stuffing everything into the prompt, production systems need to allocate tokens across competing needs — document content, conversation history, annotations, and notes — with explicit priorities and guaranteed minimums. In our system, we guarantee at least 10,000 tokens for document context no matter how long the conversation gets, and we cap annotations and notes at 12% of the total budget.</p><h3>Failure Mode #2: Imprecise Retrieval</h3><p>Standard vector similarity search sounds great in theory: embed your chunks, embed the query, find the nearest neighbors. In practice, it misses context that matters and retrieves context that doesn’t.</p><p>Here’s a scenario that illustrates the problem. A user is reading a machine learning textbook. On page 45, they highlighted a passage about gradient descent and wrote a note: “This connects to the optimization problem discussed in Chapter 2.” Later, on page 82, they ask the AI: “How does the optimization approach in this book differ from standard methods?”</p><p>Without embeddings, a proximity-based system only looks at content near page 82. The critical annotation on page 45 — which the user explicitly connected to this topic — is completely ignored. The AI gives a generic answer that misses the cross-reference the user already identified.</p><p>But even with embeddings, naive vector search has problems:</p><ol><li><strong>No positional awareness.</strong> A chunk from the user’s current reading position should get a relevance boost — the user is likely asking about what they’re reading right now.</li><li><strong>No structural understanding.</strong> When a user asks “What does Chapter 5 say about neural networks?”, the system needs to understand document structure, not just semantic similarity. A chunk from Chapter 5 with moderate similarity should outrank a highly similar chunk from Chapter 12.</li><li><strong>No distinction between content types.</strong> User annotations carry intent — the user thought something was important enough to highlight. These should be weighted differently than random document chunks.</li><li><strong>The fix: Multi-signal retrieval.</strong> Production systems need to combine vector similarity with positional signals, structural hints, and content-type weighting. In our system, annotations on the current page get a +0.15 relevance boost, adjacent pages get +0.05, and annotations with user notes get an additional +0.03. When the user asks about a specific chapter, chunks from that chapter get a +0.3 structural boost. These small adjustments dramatically improve retrieval quality.</li><li>We also implement a three-path retrieval strategy:</li></ol><ul><li><strong>Path A (optimal):</strong> Vector retrieval selects the top 5 most relevant chunks using combined scoring</li><li><strong>Path B (structural fallback):</strong> When the query targets specific chapters, extract content from those chapters directly</li><li><strong>Path C (safety net):</strong> For small documents or when embeddings aren’t available, send the full document or truncate to budget</li><li>This layered approach means the system always returns something useful, even when the optimal path isn’t available.</li></ul><h3>Failure Mode #3: Conversation Collapse</h3><p>This is the failure mode that catches most teams off guard. Your RAG system works great for single questions. It even works well for 5–10 turn conversations. But around message 30 or 40, something changes. The AI starts forgetting earlier context, repeating itself, or giving answers that contradict what it said ten messages ago.</p><p>The root cause is simple: conversation history grows linearly, but your token budget doesn’t.</p><p>In a 50-message conversation where each message averages 400 tokens, the history alone consumes 20,000 tokens. That’s already more than the entire effective budget for some providers. The system is forced to choose: sacrifice document context to keep conversation history, or sacrifice history to keep document context. Either way, quality degrades.</p><p>Most implementations handle this with a sliding window — keep the last N messages, drop everything else. This solves the token problem but creates a memory problem. The AI loses track of what was discussed early in the conversation. Key decisions, important clarifications, and the original intent of the conversation all vanish.</p><p><strong>The fix: Hierarchical summarization.</strong> Instead of a binary keep-or-drop approach, production systems need tiered memory:</p><p>TierCoverageCompressionDetail LevelLong-term summaryEarliest messages (1–30+)HighCore topics, decisions, goalsShort-term summaryMiddle messages (30–70)ModerateSpecific questions, key factsRecent windowLast 10 messagesNoneFull verbatim content</p><p>This architecture supports conversations of unlimited length. As the conversation grows:</p><ul><li>At <strong>40 messages</strong>, the system creates a short-term summary of messages 1–30, keeping only the last 10 in full</li><li>At <strong>70 messages</strong>, the short-term summary gets promoted to a compressed long-term summary, and a new short-term summary covers the recent middle portion</li><li>Beyond 70 messages, the cycle continues — the AI always has access to the full arc of the conversation through summaries, plus the immediate context through the recent window</li><li>There’s also a token-pressure trigger: if conversation history exceeds 80% of its allocated budget, summarization fires immediately regardless of message count. This catches edge cases where individual messages are unusually long.</li><li>The result? A conversation at message 100 has the same quality as a conversation at message 10. The AI knows what was discussed at the beginning (via long-term summary), what happened recently (via short-term summary), and what’s happening right now (via the last 10 messages).</li></ul><h3>What Production RAG Actually Requires</h3><p>These three failure modes — token explosion, imprecise retrieval, and conversation collapse — aren’t independent problems. They interact:</p><ul><li><strong>Long conversations</strong> squeeze the document budget, making retrieval precision more critical</li><li><strong>Imprecise retrieval</strong> wastes tokens on irrelevant content, accelerating token pressure</li><li><strong>Token pressure</strong> forces aggressive summarization, which can lose important context</li><li>A production RAG system needs an orchestration layer — what I call <strong>context assembly</strong> — that manages these competing concerns holistically. This isn’t just a retrieval pipeline; it’s a budget-aware, structure-aware, history-aware system that makes trade-offs in real time.</li><li>Here’s what the architecture looks like at a high level:</li></ul><pre>User Question<br>     │<br>     ▼<br>┌─────────────────────────────┐<br>│   1. Token Budget Allocation │ ← Provider-aware (200K/128K/32K)<br>│   2. Structural Classification│ ← &quot;Is this about a specific chapter?&quot;<br>│   3. Annotation Retrieval     │ ← Vector + positional + type signals<br>│   4. Note Retrieval           │ ← Vector + priority weighting<br>│   5. Document Strategy        │ ← Full / Truncated / Chunked<br>│   6. History Strategy         │ ← Full / Windowed / Summarized<br>│   7. Final Assembly           │ ← Ordered for cache efficiency<br>└─────────────────────────────┘<br>     │<br>     ▼<br>  LLM Prompt</pre><p>Each step depends on the outputs of previous steps. Token budgets constrain retrieval. Retrieval results influence document strategy. Document strategy affects how much room is left for history. It’s a pipeline, not a single-shot operation.</p><h3>What’s Next</h3><p>In the rest of this series, we’ll go deep into each of these components:</p><ul><li><strong>Part 2: Context Assembly</strong> — The 8-step pipeline that orchestrates document retrieval, annotation selection, and history compression into a single coherent prompt</li><li><strong>Part 3: Smart Caching</strong> — Three-tier caching architecture (memory → disk → API) that cuts costs by 90% and latency by 10x</li><li><strong>Part 4: Conversation Memory</strong> — Hierarchical summarization that makes 100-message conversations work as well as 10-message ones</li><li><strong>Part 5: Privacy-First Embeddings</strong> — On-device vs. cloud embeddings, and why the best RAG systems give users a choice</li><li>Each article will include concrete numbers, architectural diagrams, and implementation trade-offs from building a production system that handles documents ranging from 5 to 500 pages.</li><li>If you’re building an AI application that goes beyond single-shot Q&amp;A — anything with documents, conversations, or accumulated context — this series is for you.</li></ul><p><em>This is Part 1 of “Building AI Agents That Actually Work.” Follow along for the next installment on context assembly pipelines.</em></p><p><em>I’m building </em><a href="https://apps.apple.com/cn/app/tt-notebook/id6758238388?mt=12"><em>TT Notebook</em></a><em>, an AI-powered reading assistant for macOS that implements everything in this series. Questions or ideas? Reach me at </em><a href="/service/mailto:support@tinytwo.me"><em>support@tinytwo.me</em></a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=36fa8b7af00a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Production Ready: AI-Assisted Pre-Launch Checklist]]></title>
            <link>https://medium.com/@guqung6/production-ready-ai-assisted-pre-launch-checklist-a44858801f40?source=rss-e1958981e79b------2</link>
            <guid isPermaLink="false">https://medium.com/p/a44858801f40</guid>
            <category><![CDATA[vibe-coding]]></category>
            <category><![CDATA[deployment]]></category>
            <category><![CDATA[claude]]></category>
            <category><![CDATA[k8s]]></category>
            <category><![CDATA[kubernetes]]></category>
            <dc:creator><![CDATA[TinyTwo]]></dc:creator>
            <pubDate>Mon, 16 Feb 2026 13:01:01 GMT</pubDate>
            <atom:updated>2026-02-16T13:01:01.841Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*21GhVenY3_XAcQf315vMIw.jpeg" /></figure><h3>Preface: Pre-Launch Anxiety</h3><p>I still remember my first K8s application deployment: revising configuration files dozens of times, repeatedly checking every parameter, terrified of missing something that could cause a production incident. That anxiety — every engineer knows it.</p><p>This time, I tried a new approach: <strong>having Claude review my pre-launch setup</strong>. The results were surprising:</p><blockquote><strong><em>Me</em></strong><em>: “Claude, please review this Deployment configuration for production pitfalls.”</em></blockquote><blockquote><strong><em>Claude</em></strong><em>: “Found 3 areas needing improvement:</em></blockquote><ul><li>❌ No PodDisruptionBudget set — all Pods could be evicted during Node maintenance</li><li>❌ requests and limits are identical — cannot handle traffic bursts</li><li>⚠️ Readiness probe initialDelaySeconds is only 5 seconds — application startup may need more time”</li></ul><p><strong>Claude discovered issues I hadn’t even thought of!</strong> This article shares my <strong>AI-assisted launch review process</strong> and reusable checklists.</p><h3>AI-Assisted Review Process</h3><h3>Step 1: Prepare Prompt Template</h3><p>My prompt template for Claude:</p><pre>Please review this K8s configuration file from the following 6 dimensions and provide improvement suggestions:<br>​<br>1. Security: Secret management, permission control, image security<br>2. Reliability: Replica count, health checks, resource limits, PDB<br>3. Observability: Logging, monitoring, tracing<br>4. Performance: Resource configuration, HPA, affinity<br>5. Cost: Resource utilization, Spot instances<br>6. Best Practices: Naming conventions, labels, annotations<br>​<br>Configuration file:<br>&lt;paste your YAML configuration here&gt;<br>​<br>Please output in checklist format, marking severity for each issue (🔴Critical/🟡Warning/🟢Suggestion).</pre><h3>Step 2: Claude’s Sample Review Report</h3><p><strong>🔴 Critical Issues (Must Fix)</strong></p><ol><li>PodDisruptionBudget not set</li></ol><ul><li>Risk: All Pods could be evicted during Node maintenance, causing service interruption</li><li>Recommendation: Add PDB with minAvailable: 1</li></ul><p>2. Image using latest tag</p><ul><li>Risk: Cannot rollback, version uncontrolled</li><li>Recommendation: Use specific version number (e.g., 1.0.0)</li></ul><p><strong>🟡 Warnings (Should Fix)</strong></p><ol><li>requests == limits</li></ol><ul><li>Impact: Cannot handle traffic bursts, CPU hard-limited</li><li>Recommendation: Set limits to 1.5–2x requests</li></ul><p>2. Resource Quota not configured</p><ul><li>Impact: Single application could exhaust cluster resources</li><li>Recommendation: Configure ResourceQuota at Namespace level</li></ul><p><strong>🟢 Best Practice Suggestions</strong></p><ol><li>Recommend adding more labels</li></ol><ul><li>Current: app=demo-app</li><li>Recommendation: Add environment, team, version, etc.</li></ul><p>2. Recommend enabling HPA</p><ul><li>Current: Fixed 3 replicas</li><li>Recommendation: Configure HPA for auto-scaling based on CPU/memory</li></ul><h3>Six Review Dimensions Explained</h3><h3>Dimension 1: Security Review 🔒</h3><h4>Checklist</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UMd099W06V3m4IkXzzAicA.png" /></figure><h4>Best Practice Configuration</h4><pre># ========== Secret Management ==========<br>apiVersion: v1<br>kind: Secret<br>metadata:<br>  name: app-secrets<br>  namespace: production<br>type: Opaque<br>data:<br>  db-password: &lt;base64-encoded&gt;  # Use Secret, don&#39;t hardcode<br>  api-key: &lt;base64-encoded&gt;<br>---<br># ========== RBAC Configuration ==========<br>apiVersion: v1<br>kind: ServiceAccount<br>metadata:<br>  name: demo-app<br>  namespace: production<br>---<br>apiVersion: rbac.authorization.k8s.io/v1<br>kind: Role<br>metadata:<br>  name: demo-app-role<br>  namespace: production<br>rules:<br>- apiGroups: [&quot;&quot;]<br>  resources: [&quot;configmaps&quot;]<br>  verbs: [&quot;get&quot;, &quot;list&quot;]  # Least privilege: read-only ConfigMap<br>---<br>apiVersion: rbac.authorization.k8s.io/v1<br>kind: RoleBinding<br>metadata:<br>  name: demo-app-binding<br>  namespace: production<br>subjects:<br>- kind: ServiceAccount<br>  name: demo-app<br>roleRef:<br>  kind: Role<br>  name: demo-app-role<br>  apiGroup: rbac.authorization.k8s.io<br>---<br># ========== NetworkPolicy ==========<br>apiVersion: networking.k8s.io/v1<br>kind: NetworkPolicy<br>metadata:<br>  name: demo-app-network-policy<br>  namespace: production<br>spec:<br>  podSelector:<br>    matchLabels:<br>      app: demo-app<br>  policyTypes:<br>  - Ingress<br>  - Egress<br>  ingress:<br>  - from:<br>    - podSelector:  # Only allow from same namespace Pods<br>        matchLabels: {}<br>    ports:<br>    - protocol: TCP<br>      port: 8080<br>  egress:<br>  - to:<br>    - podSelector:  # Allow database access<br>        matchLabels:<br>          app: mysql<br>    ports:<br>    - protocol: TCP<br>      port: 3306<br>  - to:  # Allow DNS queries<br>    - namespaceSelector:<br>        matchLabels:<br>          name: kube-system<br>    ports:<br>    - protocol: UDP<br>      port: 53<br>---<br># ========== Pod Security ==========<br>apiVersion: v1<br>kind: Pod<br>metadata:<br>  name: demo-app<br>spec:<br>  serviceAccountName: demo-app  # Use dedicated ServiceAccount<br>  securityContext:<br>    runAsNonRoot: true  # Don&#39;t run as root<br>    runAsUser: 1000<br>    fsGroup: 1000<br>  containers:<br>  - name: app<br>    image: registry.cn-hangzhou.aliyuncs.com/demo/app:1.0.0<br>    securityContext:<br>      allowPrivilegeEscalation: false  # Prohibit privilege escalation<br>      readOnlyRootFilesystem: true  # Read-only root filesystem<br>      capabilities:<br>        drop:<br>        - ALL  # Drop all Linux capabilities</pre><h4>Dialogue with Claude: Security Check</h4><blockquote><strong><em>Me</em></strong><em>: “Claude, does this configuration have security risks?”</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xFlMGpq5Uqjvc_ysqvmGJg.png" /></figure><blockquote><strong><em>Claude</em></strong><em>: “🔴 Critical security issue!</em></blockquote><blockquote><strong><em>Risks</em></strong><em>:</em></blockquote><ul><li>Password stored in plaintext in YAML</li><li>Anyone who can execute kubectl get deployment -o yaml can see the password</li><li>Permanently exists in Git history</li></ul><blockquote><strong><em>Fix</em></strong><em>:</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vksYD9atRr0EH735EnBXNg.png" /></figure><blockquote><strong><em>Further recommendations</em></strong><em>:</em></blockquote><ul><li>Use Alibaba Cloud KMS to encrypt Secrets</li><li>Enable RBAC to restrict Secret access</li><li>Use external secret management (e.g., HashiCorp Vault)”</li></ul><h3>Dimension 2: Reliability Review 💪</h3><h4>Core Configuration</h4><pre># ========== PodDisruptionBudget (Required!) ==========<br>apiVersion: policy/v1<br>kind: PodDisruptionBudget<br>metadata:<br>  name: demo-app-pdb<br>  namespace: production<br>spec:<br>  minAvailable: 2  # Keep at least 2 Pods available (or use maxUnavailable: 1)<br>  selector:<br>    matchLabels:<br>      app: demo-app<br>---<br># ========== Deployment Reliability Configuration ==========<br>apiVersion: apps/v1<br>kind: Deployment<br>metadata:<br>  name: demo-app<br>  namespace: production<br>spec:<br>  replicas: 3  # 🟢 At least 3 replicas (high availability)<br>​<br>  strategy:<br>    type: RollingUpdate<br>    rollingUpdate:<br>      maxUnavailable: 1  # 🟢 Ensure at least 2 Pods available<br>      maxSurge: 1<br>​<br>  template:<br>    spec:<br>      # Affinity: Distribute across different Nodes<br>      affinity:<br>        podAntiAffinity:<br>          preferredDuringSchedulingIgnoredDuringExecution:<br>          - weight: 100<br>            podAffinityTerm:<br>              labelSelector:<br>                matchLabels:<br>                  app: demo-app<br>              topologyKey: kubernetes.io/hostname  # Different Nodes<br>​<br>      containers:<br>      - name: app<br>        image: registry.cn-hangzhou.aliyuncs.com/demo/app:1.0.0  # 🟢 Specific version<br>​<br>        # Health checks (detailed configuration)<br>        livenessProbe:<br>          httpGet:<br>            path: /healthz<br>            port: 8080<br>          initialDelaySeconds: 30<br>          periodSeconds: 10<br>          timeoutSeconds: 5<br>          failureThreshold: 3  # Restart only after 3 consecutive failures<br>​<br>        readinessProbe:<br>          httpGet:<br>            path: /ready<br>            port: 8080<br>          initialDelaySeconds: 10<br>          periodSeconds: 5<br>          timeoutSeconds: 3<br>          failureThreshold: 3<br>​<br>        # Startup probe (for slow-starting applications)<br>        startupProbe:<br>          httpGet:<br>            path: /healthz<br>            port: 8080<br>          initialDelaySeconds: 0<br>          periodSeconds: 10<br>          failureThreshold: 30  # Wait up to 5 minutes (30*10=300s)<br>​<br>        # Resource limits<br>        resources:<br>          requests:<br>            cpu: 200m<br>            memory: 256Mi<br>          limits:<br>            cpu: 500m  # 🟢 limits is 2.5x requests (handle traffic bursts)<br>            memory: 512Mi<br>​<br>        # Graceful shutdown<br>        lifecycle:<br>          preStop:<br>            exec:<br>              command: [&quot;/bin/sh&quot;, &quot;-c&quot;, &quot;sleep 15&quot;]  # Wait 15 seconds (finish current requests)<br>​<br>      terminationGracePeriodSeconds: 30  # Graceful shutdown timeout</pre><h4>PodDisruptionBudget Explained</h4><blockquote><strong><em>Me</em></strong><em>: “What is PodDisruptionBudget? Is it required?”</em></blockquote><blockquote><strong><em>Claude</em></strong><em>: “</em><strong><em>Required!</em></strong><em> This is a critical configuration many people overlook.</em></blockquote><blockquote><strong><em>Scenario</em></strong><em>: K8s Node needs maintenance (upgrade/reboot)</em></blockquote><blockquote><strong><em>Without PDB</em></strong><em>:</em></blockquote><blockquote><em>1. Node marked as unschedulable<br>2. All Pods on Node are evicted<br>3. If your 3 Pods are all on this Node → all go down!<br>4. Service interruption ❌</em></blockquote><blockquote><strong><em>With PDB (minAvailable: 2)</em></strong><em>:</em></blockquote><blockquote><em>1. K8s tries to evict Pods<br>2. Checks PDB: currently 3, minimum 2<br>3. Evicts 1 Pod<br>4. Waits for new Pod to start on other Nodes<br>5. After new Pod is ready, continues eviction<br>6. Throughout process, at least 2 Pods remain available ✅</em></blockquote><blockquote><strong><em>Configuration recommendations</em></strong><em>:</em></blockquote><ul><li>minAvailable: N-1 (e.g., for 3 replicas use minAvailable: 2)</li><li>Or maxUnavailable: 1 (at most 1 unavailable)</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4lAKIUk0Bgn05rGYC_GKkw.png" /></figure><h3>Dimension 3: Observability Review 📊</h3><h4>Logging Configuration</h4><pre>apiVersion: v1<br>kind: Pod<br>spec:<br>  containers:<br>  - name: app<br>    image: registry.cn-hangzhou.aliyuncs.com/demo/app:1.0.0<br>​<br>    # Environment variables: control log format<br>    env:<br>    - name: LOG_FORMAT<br>      value: &quot;json&quot;  # 🟢 Use JSON format (easy to parse)<br>    - name: LOG_LEVEL<br>      value: &quot;info&quot;  # info for production, debug for development<br>​<br>    # Standard output (auto-collected)<br>    # stdout/stderr will be collected by K8s logging system<br>​<br>  # Sidecar container: log collection (optional)<br>  - name: log-collector<br>    image: fluentd:v1.14<br>    volumeMounts:<br>    - name: varlog<br>      mountPath: /var/log<br>  volumes:<br>  - name: varlog<br>    emptyDir: {}</pre><h4>Monitoring Configuration</h4><pre># ========== ServiceMonitor (Prometheus) ==========<br>apiVersion: monitoring.coreos.com/v1<br>kind: ServiceMonitor<br>metadata:<br>  name: demo-app<br>  namespace: production<br>spec:<br>  selector:<br>    matchLabels:<br>      app: demo-app<br>  endpoints:<br>  - port: metrics  # Service metrics port<br>    interval: 30s<br>    path: /metrics<br>---<br># ========== Service with metrics port ==========<br>apiVersion: v1<br>kind: Service<br>metadata:<br>  name: demo-app<br>  labels:<br>    app: demo-app  # ServiceMonitor selects via this label<br>spec:<br>  ports:<br>  - name: http<br>    port: 8080<br>    targetPort: 8080<br>  - name: metrics  # 🟢 Add metrics port<br>    port: 9090<br>    targetPort: 9090<br>  selector:<br>    app: demo-app</pre><h4>Key Metrics</h4><pre># Application should expose metrics (Prometheus format)<br># /metrics endpoint should return:<br># 1. HTTP request count<br>http_requests_total{method=&quot;GET&quot;,path=&quot;/api/users&quot;,status=&quot;200&quot;} 12345<br># 2. HTTP request latency<br>http_request_duration_seconds_bucket{path=&quot;/api/users&quot;,le=&quot;0.1&quot;} 8000<br>http_request_duration_seconds_bucket{path=&quot;/api/users&quot;,le=&quot;0.5&quot;} 11000<br>http_request_duration_seconds_sum{path=&quot;/api/users&quot;} 1234.5<br>http_request_duration_seconds_count{path=&quot;/api/users&quot;} 12345<br># 3. Business metrics<br>active_users_total 567<br>orders_created_total 123<br># 4. Resource usage<br>process_cpu_seconds_total 789.12<br>process_resident_memory_bytes 134217728</pre><h4>Alert Rules</h4><pre># PrometheusRule example<br>apiVersion: monitoring.coreos.com/v1<br>kind: PrometheusRule<br>metadata:<br>  name: demo-app-alerts<br>  namespace: production<br>spec:<br>  groups:<br>  - name: demo-app<br>    interval: 30s<br>    rules:<br>    # Alert 1: Pod restarting frequently<br>    - alert: PodRestartingTooOften<br>      expr: rate(kube_pod_container_status_restarts_total{pod=~&quot;demo-app-.*&quot;}[15m]) &gt; 0<br>      for: 5m<br>      labels:<br>        severity: warning<br>      annotations:<br>        summary: &quot;Pod {{ $labels.pod }} restarting frequently&quot;<br>        description: &quot;Restarted {{ $value }} times in last 15 minutes&quot;<br>    # Alert 2: Response time too high<br>    - alert: HighResponseTime<br>      expr: histogram_quantile(0.95, http_request_duration_seconds_bucket{app=&quot;demo-app&quot;}) &gt; 1<br>      for: 5m<br>      labels:<br>        severity: warning<br>      annotations:<br>        summary: &quot;P95 latency exceeds 1 second&quot;<br>    # Alert 3: High error rate<br>    - alert: HighErrorRate<br>      expr: rate(http_requests_total{app=&quot;demo-app&quot;,status=~&quot;5..&quot;}[5m]) / rate(http_requests_total{app=&quot;demo-app&quot;}[5m]) &gt; 0.05<br>      for: 5m<br>      labels:<br>        severity: critical<br>      annotations:<br>        summary: &quot;Error rate exceeds 5%&quot;</pre><h3>Dimension 4: Performance Review 🚀</h3><h4>HPA Configuration (Horizontal Pod Autoscaler)</h4><pre>apiVersion: autoscaling/v2<br>kind: HorizontalPodAutoscaler<br>metadata:<br>  name: demo-app-hpa<br>  namespace: production<br>spec:<br>  scaleTargetRef:<br>    apiVersion: apps/v1<br>    kind: Deployment<br>    name: demo-app<br>minReplicas: 3   # Minimum 3<br>  maxReplicas: 10  # Maximum 10<br>  metrics:<br>  # Metric 1: CPU utilization<br>  - type: Resource<br>    resource:<br>      name: cpu<br>      target:<br>        type: Utilization<br>        averageUtilization: 70  # Scale up when CPU exceeds 70%<br>  # Metric 2: Memory utilization<br>  - type: Resource<br>    resource:<br>      name: memory<br>      target:<br>        type: Utilization<br>        averageUtilization: 80  # Scale up when memory exceeds 80%<br>  # Metric 3: Custom metric (QPS)<br>  - type: Pods<br>    pods:<br>      metric:<br>        name: http_requests_per_second<br>      target:<br>        type: AverageValue<br>        averageValue: &quot;1000&quot;  # Scale up when average QPS exceeds 1000<br>  behavior:<br>    scaleUp:<br>      stabilizationWindowSeconds: 60  # Wait 60s before scaling up (avoid flapping)<br>      policies:<br>      - type: Percent<br>        value: 50  # Scale up by max 50% at a time<br>        periodSeconds: 60<br>    scaleDown:<br>      stabilizationWindowSeconds: 300  # Wait 5 minutes before scaling down (avoid frequent scaling)<br>      policies:<br>      - type: Pods<br>        value: 1  # Scale down by max 1 Pod at a time<br>        periodSeconds: 60</pre><h4>Affinity Configuration</h4><pre>apiVersion: apps/v1<br>kind: Deployment<br>spec:<br>  template:<br>    spec:<br>      # Pod anti-affinity: Distribute across different Nodes (high availability)<br>      affinity:<br>        podAntiAffinity:<br>          # Hard requirement: must distribute<br>          requiredDuringSchedulingIgnoredDuringExecution:<br>          - labelSelector:<br>              matchLabels:<br>                app: demo-app<br>            topologyKey: kubernetes.io/hostname<br># Soft requirement: prefer distribution across different zones<br>          preferredDuringSchedulingIgnoredDuringExecution:<br>          - weight: 100<br>            podAffinityTerm:<br>              labelSelector:<br>                matchLabels:<br>                  app: demo-app<br>              topologyKey: topology.kubernetes.io/zone<br>      # Node affinity: Prefer scheduling to high-performance nodes<br>      nodeAffinity:<br>        preferredDuringSchedulingIgnoredDuringExecution:<br>        - weight: 50<br>          preference:<br>            matchExpressions:<br>            - key: node.kubernetes.io/instance-type<br>              operator: In<br>              values:<br>              - ecs.c6.2xlarge  # Prefer compute-optimized instances</pre><h4>Resource Configuration Best Practices</h4><blockquote><strong><em>Me</em></strong><em>: “Claude, how do I determine reasonable resource configuration?”</em></blockquote><blockquote><strong><em>Claude</em></strong><em>: “4 steps:</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IlypTfPafgVQ6Av8f5aSLw.png" /></figure><h3>Dimension 5: Cost Optimization 💰</h3><h4>ResourceQuota (Limit namespace resources)</h4><pre>apiVersion: v1<br>kind: ResourceQuota<br>metadata:<br>  name: production-quota<br>  namespace: production<br>spec:<br>  hard:<br>    requests.cpu: &quot;10&quot;        # Entire namespace can request max 10 cores<br>    requests.memory: &quot;20Gi&quot;   # Max request 20GB memory<br>    limits.cpu: &quot;20&quot;          # Max limit 20 cores<br>    limits.memory: &quot;40Gi&quot;<br>    pods: &quot;50&quot;                # Max 50 Pods<br>    services: &quot;20&quot;            # Max 20 Services<br>    persistentvolumeclaims: &quot;10&quot;  # Max 10 PVCs</pre><h4>LimitRange (Limit individual Pod resources)</h4><pre>apiVersion: v1<br>kind: LimitRange<br>metadata:<br>  name: production-limit-range<br>  namespace: production<br>spec:<br>  limits:<br>  # Container resource range<br>  - type: Container<br>    max:<br>      cpu: &quot;2&quot;         # Single container max 2 cores<br>      memory: &quot;4Gi&quot;    # Single container max 4GB<br>    min:<br>      cpu: &quot;50m&quot;       # Single container min 50m<br>      memory: &quot;64Mi&quot;   # Single container min 64MB<br>    default:<br>      cpu: &quot;200m&quot;      # Default limit<br>      memory: &quot;256Mi&quot;<br>    defaultRequest:<br>      cpu: &quot;100m&quot;      # Default request<br>      memory: &quot;128Mi&quot;<br>  # Pod resource range<br>  - type: Pod<br>    max:<br>      cpu: &quot;4&quot;<br>      memory: &quot;8Gi&quot;</pre><h4>Spot Instances (Preemptible instances)</h4><pre># Use Spot instances to reduce costs (suitable for stateless applications)<br>apiVersion: apps/v1<br>kind: Deployment<br>metadata:<br>  name: demo-app-spot<br>spec:<br>  replicas: 5<br>  template:<br>    spec:<br>      # Node selector: select Spot instances<br>      nodeSelector:<br>        kubernetes.io/lifecycle: spot  # Spot instance label<br># Tolerations: allow scheduling on Spot instances<br>      tolerations:<br>      - key: &quot;spot&quot;<br>        operator: &quot;Equal&quot;<br>        value: &quot;true&quot;<br>        effect: &quot;NoSchedule&quot;<br>      # Priority: lower than production applications<br>      priorityClassName: low-priority</pre><p><strong>Cost Comparison</strong>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7zIE7lwhiSO-yHXfKfydoQ.png" /></figure><h3>Dimension 6: Best Practices ✨</h3><h4>Naming Conventions</h4><pre># 🟢 Good naming<br>metadata:<br>  name: user-service-production  # {service-name}-{environment}<br>  namespace: production<br>  labels:<br>    app: user-service              # Application name<br>    environment: production        # Environment<br>    team: backend                  # Team<br>    version: v1.2.3               # Version<br>    component: api                # Component type<br>  annotations:<br>    description: &quot;User service API&quot;<br>    owner: &quot;backend-team@example.com&quot;<br>    docs: &quot;https://wiki.example.com/user-service&quot;<br># ❌ Bad naming<br>metadata:<br>  name: app1  # Meaningless name<br>  labels:<br>    app: app1  # Lacks context information</pre><h4>Label Design</h4><pre># Recommended label system<br>labels:<br>  # K8s recommended labels<br>  app.kubernetes.io/name: user-service<br>  app.kubernetes.io/instance: user-service-production<br>  app.kubernetes.io/version: &quot;1.2.3&quot;<br>  app.kubernetes.io/component: api<br>  app.kubernetes.io/part-of: e-commerce-platform<br>  app.kubernetes.io/managed-by: helm<br># Custom labels<br>  environment: production<br>  team: backend<br>  cost-center: engineering</pre><p><strong>Label usage</strong>:</p><pre># 1. Query by environment<br>kubectl get pods -l environment=production<br># 2. Query by team<br>kubectl get all -l team=backend --all-namespaces<br># 3. Cost tracking<br>kubectl get pods -l cost-center=engineering -o json | jq &#39;.items[].spec.containers[].resources.requests&#39;<br># 4. Monitoring grouping (Prometheus)<br>sum(rate(http_requests_total[5m])) by (team, environment)</pre><h3>Complete Checklist</h3><h3>📋 Pre-Launch Master Checklist</h3><pre># Use this YAML as a checklist template, confirm each item<br># ========== 1. Security ==========<br>- [ ] Secret does not contain plaintext passwords<br>- [ ] Images from trusted registry<br>- [ ] ServiceAccount configured<br>- [ ] RBAC configured (least privilege)<br>- [ ] Container does not run as root (runAsNonRoot: true)<br>- [ ] Read-only root filesystem (readOnlyRootFilesystem: true)<br>- [ ] NetworkPolicy configured (network isolation)<br>- [ ] Image scanning enabled<br># ========== 2. Reliability ==========<br>- [ ] Replicas &gt;= 3 (production environment)<br>- [ ] PodDisruptionBudget configured<br>- [ ] Liveness probe configured<br>- [ ] Readiness probe configured<br>- [ ] startupProbe configured (if slow startup)<br>- [ ] Image uses specific version (not latest)<br>- [ ] Pod anti-affinity configured (distribute across different Nodes)<br>- [ ] Graceful shutdown configured (preStop hook)<br>- [ ] terminationGracePeriodSeconds &gt;= 30<br># ========== 3. Resource Configuration ==========<br>- [ ] resources.requests set<br>- [ ] resources.limits set<br>- [ ] limits &gt;= requests * 1.5<br>- [ ] Resource configuration verified by load testing<br>- [ ] HPA configured (if auto-scaling needed)<br>- [ ] ResourceQuota configured (namespace level)<br>- [ ] LimitRange configured (namespace level)<br># ========== 4. Observability ==========<br>- [ ] Logs output to stdout/stderr<br>- [ ] Log format is JSON (easy to parse)<br>- [ ] /metrics endpoint exposed (Prometheus format)<br>- [ ] ServiceMonitor configured<br>- [ ] Alert rules configured (PrometheusRule)<br>- [ ] Key metrics set (latency, error rate, QPS)<br>- [ ] Distributed tracing configured (if needed)<br># ========== 5. Network Configuration ==========<br>- [ ] Service configured correctly (port, targetPort consistent)<br>- [ ] Ingress configured correctly (host, path, backend)<br>- [ ] TLS certificate configured correctly (if using HTTPS)<br>- [ ] Health check path matches Readiness<br>- [ ] Session affinity configured (if needed)<br>- [ ] DNS configured correctly (domain points to ALB)<br># ========== 6. Best Practices ==========<br>- [ ] Meaningful naming used<br>- [ ] Complete labels configured (app, environment, team, etc.)<br>- [ ] Annotations added (description, owner, documentation link)<br>- [ ] Configuration file passed YAML lint check<br>- [ ] Passed AI Review (Claude check)<br>- [ ] Rollback plan prepared</pre><h3>AI-Assisted Launch Process</h3><h3>Complete Process Flow</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ae36zfa5WAyeUKimHo7Jxg.png" /></figure><h3>Claude Prompts for Each Stage</h3><h4>Stage 1: Security Review</h4><pre>Please review the security of this K8s configuration:<br>1. Are there any hardcoded passwords/keys?<br>2. Is the container running as root?<br>3. Is least privilege configured (RBAC)?<br>4. Does the image use a specific version?<br>5. Is NetworkPolicy configured?<br>Configuration file:<br>&lt;paste your YAML configuration here&gt;<br>Please categorize as 🔴Critical/🟡Warning/🟢Suggestion.</pre><h4>Stage 2: Reliability Review</h4><pre>Please review the reliability of this K8s configuration:<br>1. Are there enough replicas? (at least 3 for production)<br>2. Is PodDisruptionBudget configured?<br>3. Are health checks comprehensive? (Liveness + Readiness)<br>4. Are resource configurations reasonable? (requests and limits)<br>5. Is Pod anti-affinity configured?<br>Configuration file:<br>&lt;paste your YAML configuration here&gt;<br>If issues found, please provide the corrected complete YAML.</pre><h4>Stage 3: Performance Review</h4><pre>Please review the performance of this K8s configuration:<br>1. Is HPA configuration reasonable?<br>2. Are resource requests/limits reasonable?<br>3. Is affinity configured (reduce cross-Node communication)?<br>4. Is Service&#39;s externalTrafficPolicy reasonable?<br>My scenario:<br>- Expected QPS: 10000<br>- Single Pod capacity: 500 QPS<br>- Latency requirement: P99 &lt; 100ms<br>Configuration file:<br>&lt;paste your YAML configuration here&gt;<br>Please provide optimization suggestions.</pre><h3>Real Cases: Issues Found by AI</h3><h3>Case 1: Missing PDB</h3><p>My configuration:</p><pre>apiVersion: apps/v1<br>kind: Deployment<br>metadata:<br>  name: payment-service<br>spec:<br>  replicas: 3<br>  # ... other configuration</pre><p>Claude’s feedback:</p><blockquote><em>🔴 </em><strong><em>Critical Issue</em></strong><em>: Missing PodDisruptionBudget</em></blockquote><blockquote><strong><em>Risk scenario</em></strong><em>:</em></blockquote><ul><li>During Node upgrade, K8s may evict all Pods simultaneously</li><li>Causing payment service to become completely unavailable</li><li>Affecting transactions, causing financial loss</li></ul><blockquote><strong><em>Fix</em></strong><em>:</em></blockquote><blockquote><em>apiVersion: policy/v1<br>kind: PodDisruptionBudget<br>metadata:<br> name: payment-service-pdb<br>spec:<br> minAvailable: 2 # Keep at least 2 available<br> selector:<br> matchLabels:<br> app: payment-service</em></blockquote><p><strong>Consequence</strong>: Without Claude’s reminder, this could have caused payment service interruption during Node maintenance!</p><h3>Case 2: Improper Resource Configuration</h3><p>My configuration:</p><pre>resources:<br>  requests:<br>    cpu: 100m<br>    memory: 128Mi<br>  limits:<br>    cpu: 100m      # ❌ Same as requests<br>    memory: 128Mi  # ❌ Same as requests</pre><p>Claude’s feedback:</p><blockquote><em>🟡 </em><strong><em>Warning</em></strong><em>: requests == limits, QoS is Guaranteed</em></blockquote><blockquote><strong><em>Issues</em></strong><em>:</em></blockquote><ul><li>CPU is hard-limited, cannot handle traffic bursts (CPU throttling)</li><li>Memory exceeding 128Mi leads to immediate OOMKilled, no buffer</li></ul><blockquote><strong><em>Recommendation</em></strong><em>:</em></blockquote><blockquote><em>resources:<br> requests:<br> cpu: 100m<br> memory: 128Mi<br> limits:<br> cpu: 200m # 2x, allow burst<br> memory: 256Mi # 2x, buffer space</em></blockquote><blockquote><strong><em>Scenario</em></strong><em>:</em></blockquote><ul><li>Normal CPU usage 100m (meets requests)</li><li>During traffic spike CPU can use up to 200m (limits)</li><li>Improves user experience, avoids throttling</li></ul><h3>Summary: Value of AI Assistance</h3><p>Through this 5-article series, I grew from a K8s novice to an engineer capable of independently deploying production applications. <strong>AI played a key role</strong>:</p><h3>🎯 AI’s Value</h3><p>Traditional LearningAI-Assisted ApproachImprovementRead docs, summarize yourselfAsk directly, AI summarizes⏰ Save 80% timeSearch multiple sources for problemsAI instant answers💡 Improved accuracyConfiguration based on experience, easy to missAI review, discover blind spots🛡️ Improved reliabilityOnly learn about mistakes after incidentsAI early warning🔮 Risk prevention</p><h3>🚀 Next Steps</h3><p>Continue using AI to learn advanced K8s topics:</p><ul><li>StatefulSet (stateful applications)</li><li>DaemonSet (node daemons)</li><li>CronJob (scheduled tasks)</li><li>Operator (custom controllers)</li><li>Service Mesh (Istio)</li></ul><p><strong>Prompt template</strong>:</p><pre>I want to learn about K8s [topic], please:<br>1. Explain core concepts using analogies<br>2. Provide complete YAML examples (anonymized)<br>3. Draw architecture diagrams with Mermaid<br>4. List common pitfalls and best practices<br>5. Provide troubleshooting command checklists</pre><h3>Appendix: Complete Production-Grade Configuration Template</h3><pre># ========== Namespace ==========<br>apiVersion: v1<br>kind: Namespace<br>metadata:<br>  name: production<br>  labels:<br>    environment: production<br>---<br># ========== ResourceQuota ==========<br>apiVersion: v1<br>kind: ResourceQuota<br>metadata:<br>  name: production-quota<br>  namespace: production<br>spec:<br>  hard:<br>    requests.cpu: &quot;10&quot;<br>    requests.memory: &quot;20Gi&quot;<br>    limits.cpu: &quot;20&quot;<br>    limits.memory: &quot;40Gi&quot;<br>    pods: &quot;50&quot;<br>---<br># ========== LimitRange ==========<br>apiVersion: v1<br>kind: LimitRange<br>metadata:<br>  name: production-limit-range<br>  namespace: production<br>spec:<br>  limits:<br>  - type: Container<br>    max:<br>      cpu: &quot;2&quot;<br>      memory: &quot;4Gi&quot;<br>    min:<br>      cpu: &quot;50m&quot;<br>      memory: &quot;64Mi&quot;<br>    default:<br>      cpu: &quot;200m&quot;<br>      memory: &quot;256Mi&quot;<br>    defaultRequest:<br>      cpu: &quot;100m&quot;<br>      memory: &quot;128Mi&quot;<br>---<br># ========== ServiceAccount ==========<br>apiVersion: v1<br>kind: ServiceAccount<br>metadata:<br>  name: demo-app<br>  namespace: production<br>---<br># ========== ConfigMap ==========<br>apiVersion: v1<br>kind: ConfigMap<br>metadata:<br>  name: demo-app-config<br>  namespace: production<br>data:<br>  APP_ENV: &quot;production&quot;<br>  LOG_LEVEL: &quot;info&quot;<br>  LOG_FORMAT: &quot;json&quot;<br>---<br># ========== Deployment ==========<br>apiVersion: apps/v1<br>kind: Deployment<br>metadata:<br>  name: demo-app<br>  namespace: production<br>  labels:<br>    app: demo-app<br>    environment: production<br>    team: backend<br>  annotations:<br>    description: &quot;Demo application&quot;<br>    owner: &quot;backend-team@example.com&quot;<br>spec:<br>  replicas: 3<br>​<br>  strategy:<br>    type: RollingUpdate<br>    rollingUpdate:<br>      maxUnavailable: 1<br>      maxSurge: 1<br>​<br>  selector:<br>    matchLabels:<br>      app: demo-app<br>​<br>  template:<br>    metadata:<br>      labels:<br>        app: demo-app<br>        environment: production<br>        version: v1.0.0<br>    spec:<br>      serviceAccountName: demo-app<br>​<br>      affinity:<br>        podAntiAffinity:<br>          requiredDuringSchedulingIgnoredDuringExecution:<br>          - labelSelector:<br>              matchLabels:<br>                app: demo-app<br>            topologyKey: kubernetes.io/hostname<br>​<br>      securityContext:<br>        runAsNonRoot: true<br>        runAsUser: 1000<br>        fsGroup: 1000<br>​<br>      containers:<br>      - name: app<br>        image: registry.cn-hangzhou.aliyuncs.com/demo/app:1.0.0<br>        imagePullPolicy: IfNotPresent<br>​<br>        ports:<br>        - name: http<br>          containerPort: 8080<br>        - name: metrics<br>          containerPort: 9090<br>​<br>        envFrom:<br>        - configMapRef:<br>            name: demo-app-config<br>​<br>        resources:<br>          requests:<br>            cpu: 200m<br>            memory: 256Mi<br>          limits:<br>            cpu: 500m<br>            memory: 512Mi<br>​<br>        livenessProbe:<br>          httpGet:<br>            path: /healthz<br>            port: 8080<br>          initialDelaySeconds: 30<br>          periodSeconds: 10<br>          timeoutSeconds: 5<br>          failureThreshold: 3<br>​<br>        readinessProbe:<br>          httpGet:<br>            path: /ready<br>            port: 8080<br>          initialDelaySeconds: 10<br>          periodSeconds: 5<br>          timeoutSeconds: 3<br>          failureThreshold: 3<br>​<br>        startupProbe:<br>          httpGet:<br>            path: /healthz<br>            port: 8080<br>          initialDelaySeconds: 0<br>          periodSeconds: 10<br>          failureThreshold: 30<br>​<br>        lifecycle:<br>          preStop:<br>            exec:<br>              command: [&quot;/bin/sh&quot;, &quot;-c&quot;, &quot;sleep 15&quot;]<br>​<br>        securityContext:<br>          allowPrivilegeEscalation: false<br>          readOnlyRootFilesystem: true<br>          capabilities:<br>            drop:<br>            - ALL<br>​<br>      terminationGracePeriodSeconds: 30<br>---<br># ========== Service ==========<br>apiVersion: v1<br>kind: Service<br>metadata:<br>  name: demo-app<br>  namespace: production<br>  labels:<br>    app: demo-app<br>spec:<br>  type: ClusterIP<br>  selector:<br>    app: demo-app<br>  ports:<br>  - name: http<br>    port: 8080<br>    targetPort: 8080<br>  - name: metrics<br>    port: 9090<br>    targetPort: 9090<br>---<br># ========== PodDisruptionBudget ==========<br>apiVersion: policy/v1<br>kind: PodDisruptionBudget<br>metadata:<br>  name: demo-app-pdb<br>  namespace: production<br>spec:<br>  minAvailable: 2<br>  selector:<br>    matchLabels:<br>      app: demo-app<br>---<br># ========== HPA ==========<br>apiVersion: autoscaling/v2<br>kind: HorizontalPodAutoscaler<br>metadata:<br>  name: demo-app-hpa<br>  namespace: production<br>spec:<br>  scaleTargetRef:<br>    apiVersion: apps/v1<br>    kind: Deployment<br>    name: demo-app<br>  minReplicas: 3<br>  maxReplicas: 10<br>  metrics:<br>  - type: Resource<br>    resource:<br>      name: cpu<br>      target:<br>        type: Utilization<br>        averageUtilization: 70<br>  - type: Resource<br>    resource:<br>      name: memory<br>      target:<br>        type: Utilization<br>        averageUtilization: 80<br>  behavior:<br>    scaleUp:<br>      stabilizationWindowSeconds: 60<br>      policies:<br>      - type: Percent<br>        value: 50<br>        periodSeconds: 60<br>    scaleDown:<br>      stabilizationWindowSeconds: 300<br>      policies:<br>      - type: Pods<br>        value: 1<br>        periodSeconds: 60<br>---<br># ========== Ingress ==========<br>apiVersion: networking.k8s.io/v1<br>kind: Ingress<br>metadata:<br>  name: demo-app-ingress<br>  namespace: production<br>  annotations:<br>    alb.ingress.kubernetes.io/healthcheck-enabled: &quot;true&quot;<br>    alb.ingress.kubernetes.io/healthcheck-path: &quot;/healthz&quot;<br>    alb.ingress.kubernetes.io/ssl-redirect: &quot;true&quot;<br>spec:<br>  ingressClassName: alb<br>  rules:<br>  - host: api.example.com<br>    http:<br>      paths:<br>      - path: /<br>        pathType: Prefix<br>        backend:<br>          service:<br>            name: demo-app<br>            port:<br>              number: 8080<br>  tls:<br>  - hosts:<br>    - api.example.com<br>    secretName: example-tls</pre><p><strong>Usage</strong>:</p><ol><li>Replace all demo-app with your application name</li><li>Replace image address</li><li>Adjust resource configuration (requests/limits)</li><li>Configure domain and certificate</li><li>Execute: kubectl apply -f production.yaml</li></ol><p><strong>Author</strong>: [TinyTwo] <strong>Creation Tools</strong>: This article was created with assistance from Claude AI, all flowcharts drawn with Mermaid <strong>Series Articles</strong>: [5/5] — Final Chapter</p><blockquote><em>💡 </em><strong><em>Series Summary</em></strong><em>: From zero to production deployment, 5 articles, 5 days. This is the power of AI-assisted learning!</em></blockquote><blockquote><em>🙏 </em><strong><em>Acknowledgments</em></strong><em>: Thanks to Claude AI for assistance throughout the learning and deployment process, making complex K8s accessible.</em></blockquote><blockquote><em>📬 </em><strong><em>Feedback</em></strong><em>: If this series has helped you, please share it with more developers. For any questions, continue asking Claude!</em></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a44858801f40" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Architecture Principles: Deep Dive into Dual-Layer Load Balancing]]></title>
            <link>https://medium.com/@guqung6/architecture-principles-deep-dive-into-dual-layer-load-balancing-efb1bb455db1?source=rss-e1958981e79b------2</link>
            <guid isPermaLink="false">https://medium.com/p/efb1bb455db1</guid>
            <category><![CDATA[deployment]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[claude]]></category>
            <category><![CDATA[k8s]]></category>
            <category><![CDATA[vibe-coding]]></category>
            <dc:creator><![CDATA[TinyTwo]]></dc:creator>
            <pubDate>Mon, 09 Feb 2026 13:01:05 GMT</pubDate>
            <atom:updated>2026-02-09T13:01:05.478Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zvm7uaZ_lZRvQjO8zXhdCQ.jpeg" /></figure><h3>Preface: The Beginning of Confusion</h3><p>When I first saw the K8s network architecture diagram, my mind was filled with confusion:</p><blockquote><strong><em>Me</em></strong><em>: “Claude, why do we need both ALB and Service for load balancing? It seems very complex. Can’t we just use ALB directly to Pods?”</em></blockquote><p>This question took Claude an hour to help me fully understand. This article will <strong>completely recreate our conversation process</strong>, taking you from confusion to enlightenment.</p><h3>Step One: Breaking Cognitive Misconceptions</h3><h3>Misconception 1: Is Service a Load Balancer?</h3><blockquote><strong><em>Me</em></strong><em>: “Isn’t Service just K8s’s load balancer?”</em></blockquote><blockquote><strong><em>Claude</em></strong><em>: “</em><strong><em>This is the biggest misconception!</em></strong><em> Service is </em><strong><em>not</em></strong><em> a load balancer; it’s just a </em><strong><em>configuration object</em></strong><em>.</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GLPMcRzRyLpKUKtUe67kDQ.png" /></figure><p>I was shocked and immediately verified it on the server:</p><pre># Execute on K8s Node<br>ps aux | grep service<br># Result: No service-related processes!<br>​<br># Then who is doing the load balancing?<br>ps aux | grep kube-proxy<br># Found it! kube-proxy process exists</pre><h3>Misconception 2: Is kube-proxy a Load Balancer?</h3><blockquote><strong><em>Me</em></strong><em>: “So is kube-proxy doing the load balancing?”</em></blockquote><blockquote><strong><em>Claude</em></strong><em>: “Close, but not accurate. </em><strong><em>kube-proxy is a rule generator</em></strong><em>; the actual load balancing is performed by </em><strong><em>iptables or IPVS</em></strong><em>.</em></blockquote><blockquote><em>Complete role division:</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*j86tnLfXvl_WczOsN1QDIQ.png" /></figure><blockquote><strong><em>Analogy</em></strong><em>:</em></blockquote><ul><li>Service = Building blueprint (defines ‘what is needed’)</li><li>kube-proxy = Construction team (reads blueprint, generates construction rules)</li><li>iptables/IPVS = Construction workers (actually does the work)</li></ul><p>I drew a diagram to understand:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XHl0IUGrys1BwS8VPVrGgQ.png" /></figure><h3>Step Two: Why Two Layers Are Needed?</h3><h3>Problems with Single-Layer Solutions</h3><blockquote><strong><em>Me</em></strong><em>: “Since Service is implemented through iptables, why not directly use ALB to Pods?”</em></blockquote><blockquote><strong><em>Claude</em></strong><em>: “Good question! Let’s see what would happen with only one layer:</em></blockquote><p><strong>Solution A: Only ALB, directly to Pods</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*M6NMKu-LeqqyandzetcBJg.png" /></figure><p><strong>Problems</strong>:</p><ol><li>❌ <strong>Pod IPs change frequently</strong></li></ol><ul><li>Pod IPs change on restart/rolling updates</li><li>ALB needs to track Pod IPs in real-time (almost impossible)</li></ul><p>2. ❌ <strong>Cross-Node communication issues</strong></p><ul><li>Pod IPs are virtual IPs, only reachable within K8s network</li><li>ALB cannot route directly to Pod IPs</li></ul><p>3. ❌ <strong>Inaccurate health checks</strong></p><ul><li>Pod may be Running but application not ready</li><li>ALB health checks cannot determine Pod readiness status</li></ul><p>4. ❌ <strong>Cannot support inter-microservice calls</strong></p><ul><li>How should Service A call Service B?</li><li>Cannot configure ALB for every service</li></ul><p><strong>Solution B: Only Service, no ALB</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GxCMFhdEwbGB_TAA05CsqA.png" /></figure><p><strong>Problems</strong>:</p><ol><li>❌ <strong>ClusterIP only accessible inside cluster</strong></li></ol><ul><li>Users are on the public internet, cannot access 10.0.100.50</li></ul><p>2. ❌ <strong>NodePort port conflicts</strong></p><ul><li>Using NodePort for exposure, limited port range (30000–32767)</li><li>100 services need 100 ports</li></ul><p>3. ❌ <strong>Lack of Layer 7 routing</strong></p><ul><li>Cannot route based on domain/path</li><li>Cannot perform TLS termination</li></ul><p><strong>Correct Solution: Combining Two Layers</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bRmh6MKMQ_ca-PMM46jh7Q.png" /></figure><p><strong>Advantages</strong>:</p><p>✅ <strong>First Layer (ALB)</strong>: Cross-Node load balancing, Layer 7 routing, TLS termination</p><p>✅ <strong>Second Layer (Service/iptables)</strong>: Cross-Pod load balancing, automatically tracks Pod changes</p><p>✅ <strong>Decoupling</strong>: ALB only needs to know Node IPs (fixed), doesn’t care about Pod IPs (volatile)</p><p>✅ <strong>High Availability</strong>: If Node fails, ALB automatically switches; if Pod fails, iptables automatically switches</p><h3>Step Three: Complete Packet Journey</h3><p>I asked Claude: “Can you explain in detail how a request reaches a Pod?”</p><h3>7-Step Complete Flow</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QC-ddA_4V7nRxU8znAq0NQ.png" /></figure><h4>Steps 1–2: Domain Resolution and HTTPS Request</h4><pre># User side (simulated with curl)<br>curl https://api.example.com/v1/users<br>​<br># 1. DNS resolution<br># api.example.com → alb-xxxxx.cn-hangzhou.alb.aliyuncs.com → 47.98.123.45<br>​<br># 2. Establish TLS connection<br># Client Hello → Server Hello → Certificate verification → Key exchange<br>​<br># 3. Send HTTPS request<br># GET /v1/users HTTP/1.1<br># Host: api.example.com</pre><h4>Steps 3–4: ALB Processing (First Layer Load Balancing)</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7vMGvbV6BCq3wTYkRhb3lQ.png" /></figure><p><strong>Key Points</strong>:</p><ul><li><strong>TLS Termination</strong>: ALB decrypts HTTPS, then uses HTTP (improves performance)</li><li><strong>Layer 7 Routing</strong>: Selects backend based on domain + path</li><li><strong>Health Checks</strong>: Only forwards to healthy Nodes</li><li><strong>Load Balancing</strong>: Round-robin/weighted round-robin/least connections algorithms</li></ul><h4>Steps 5–7: Node Processing (Second Layer Load Balancing)</h4><p>On the Node, I captured packets with tcpdump:</p><pre># Execute on Node 192.168.1.10<br>tcpdump -i eth0 port 31234 -nn<br>​<br># Captured packets:<br># 1. Inbound: 47.98.123.45 (ALB) → 192.168.1.10:31234<br># 2. After iptables processing: 192.168.1.10 → 10.244.2.8:8080</pre><p><strong>iptables Rule Chain</strong>:</p><pre>PREROUTING (pre-routing processing)<br>   ↓<br>Match KUBE-SERVICES chain<br>   ↓<br>Match KUBE-SVC-XXXXX (Service ClusterIP rules)<br>   ↓<br>Load balancing (through probability matching)<br>   ├─ 33% probability → KUBE-SEP-AAAA (Pod 1: 10.244.1.5)<br>   ├─ 33% probability → KUBE-SEP-BBBB (Pod 2: 10.244.2.8) ✅ Selected<br>   └─ 34% probability → KUBE-SEP-CCCC (Pod 3: 10.244.1.9)<br>   ↓<br>DNAT: Change destination IP to 10.244.2.8:8080<br>   ↓<br>FORWARD (forward to Pod)</pre><p><strong>Actual iptables Rules Example</strong> (simplified):</p><pre># View rules on Node<br>iptables -t nat -L KUBE-SERVICES -n<br>​<br># Service entry rule<br>-A KUBE-SERVICES -d 10.0.100.50/32 -p tcp --dport 8080 -j KUBE-SVC-XXXXX<br>​<br># Service load balancing rules<br>-A KUBE-SVC-XXXXX -m statistic --mode random --probability 0.33 -j KUBE-SEP-AAAA<br>-A KUBE-SVC-XXXXX -m statistic --mode random --probability 0.50 -j KUBE-SEP-BBBB<br>-A KUBE-SVC-XXXXX -j KUBE-SEP-CCCC<br>​<br># Pod1 DNAT rule<br>-A KUBE-SEP-AAAA -p tcp -m tcp -j DNAT --to-destination 10.244.1.5:8080<br>​<br># Pod2 DNAT rule (this one selected)<br>-A KUBE-SEP-BBBB -p tcp -m tcp -j DNAT --to-destination 10.244.2.8:8080<br>​<br># Pod3 DNAT rule<br>-A KUBE-SEP-CCCC -p tcp -m tcp -j DNAT --to-destination 10.244.1.9:8080</pre><p><strong>Load Balancing Algorithm Explanation</strong>:</p><ul><li>1st rule: 33% probability jump to Pod1</li><li>2nd rule: 50% of remaining 67% (i.e., 33% overall) jump to Pod2</li><li>3rd rule: Remaining 34% jump to Pod3</li><li><strong>Result</strong>: Traffic evenly distributed to 3 Pods</li></ul><h4>Step 8: Response Return</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FpzNIbR1Z3eUBLUhO8xTSA.png" /></figure><h3>Step Four: Real-World Scenario Analysis</h3><h3>Scenario 1: E-commerce Flash Sale (High Concurrency)</h3><blockquote><strong><em>Me</em></strong><em>: “In a Double 11 flash sale scenario, how does dual-layer load balancing cope?”</em></blockquote><blockquote><strong><em>Claude</em></strong><em>: “Let’s simulate a 100,000 QPS flash sale scenario:</em></blockquote><p><strong>Architecture Configuration</strong>:</p><pre># Deployment<br>replicas: 50  # 50 Pods<br>​<br># HPA (Horizontal Pod Autoscaler)<br>minReplicas: 50<br>maxReplicas: 200<br>targetCPUUtilizationPercentage: 70<br>​<br># Service<br>sessionAffinity: None  # No session affinity (even distribution)<br>​<br># Ingress (ALB configuration)<br>annotations:<br>  alb.ingress.kubernetes.io/backend-keepalive: &quot;true&quot;  # Enable keep-alive<br>  alb.ingress.kubernetes.io/connection-idle-timeout: &quot;60&quot;</pre><p><strong>Traffic Distribution</strong>:</p><pre>100,000 QPS<br>   ↓ ALB (First Layer)<br>Evenly distributed to 3 Nodes (each Node 33,000 QPS)<br>   ↓ iptables (Second Layer)<br>Evenly distributed to 50 Pods (each Pod 2000 QPS)<br>   ↓ HPA triggers<br>CPU exceeds 70%, scales to 100 Pods<br>   ↓<br>Each Pod reduced to 1000 QPS ✅ Stable</pre><p><strong>Value of Dual-Layer Load Balancing</strong>:</p><ol><li><strong>ALB</strong>: Quickly distributes to multiple Nodes (avoids single Node bottleneck)</li><li><strong>iptables</strong>: Automatically tracks Pod scaling (HPA scales to 100 Pods, iptables rules automatically update)</li><li><strong>Decoupling</strong>: ALB unaware of Pod count changes, configuration remains stable</li></ol><h3>Scenario 2: Rolling Update (Zero Downtime)</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lkVO9vQUUbgpfliZd0aJmg.png" /></figure><p><strong>Keys to Zero Downtime</strong>:</p><ol><li><strong>Readiness Probes</strong>: New Pods only added to iptables after ready</li><li><strong>Graceful Shutdown</strong>: Old Pods wait 30 seconds after SIGTERM (finish existing requests)</li><li><strong>Rolling Update</strong>: maxUnavailable=1, ensures at least 2 Pods available</li><li><strong>Automatic Updates</strong>: iptables rules automatically sync with Pod changes</li></ol><h3>Scenario 3: Node Failure (Automatic Failover)</h3><pre># Initial state<br>Node1: Normal, has Pod1, Pod2<br>Node2: Normal, has Pod3<br>ALB Health Check: Node1 (healthy), Node2 (healthy)<br>​<br># ========== Node1 Network Failure ==========<br># After 10 seconds, ALB health check fails (3 consecutive failures)<br>ALB: Mark Node1 as unhealthy, remove from pool<br>​<br># All new requests forwarded to Node2<br>ALB → Node2 → iptables → Pod3<br>​<br># ========== K8s Auto Recovery ==========<br># Deployment detects Pods on Node1 unreachable<br># Rebuild Pod1, Pod2 on Node2<br>​<br># After 2 minutes<br>Node2: Has Pod1, Pod2, Pod3 (3 Pods)<br>iptables rules automatically updated, traffic evenly distributed to 3 Pods<br>​<br># ========== Node1 Recovery ==========<br># Node1 network recovered<br>ALB health check successful, re-added to pool<br>​<br># Traffic redistributed<br>ALB → Node1 (50%) or Node2 (50%)</pre><p><strong>High Availability Guarantee</strong>:</p><ul><li><strong>ALB Health Checks</strong>: Automatically remove failed Nodes</li><li><strong>K8s Scheduling</strong>: Automatically rebuild Pods on healthy Nodes</li><li><strong>iptables Auto-update</strong>: No manual intervention needed</li></ul><h3>Step Five: Performance Comparison</h3><h3>iptables vs IPVS</h3><blockquote><strong><em>Me</em></strong><em>: “I heard iptables has poor performance? What should we use in production?”</em></blockquote><blockquote><strong><em>Claude</em></strong><em>: “K8s supports 3 kube-proxy modes:</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*161UsFmGM7Z3CFTOZnsVwA.png" /></figure><p><strong>Performance Comparison</strong>:</p><pre>Test scenario: 10,000 Services, each with 3 Pods<br>​<br>iptables mode:<br>- Number of rules: 30,000+ rules<br>- New connection latency: ~15ms<br>- Rule update time: ~5s<br>​<br>IPVS mode:<br>- Number of rules: 10,000 virtual servers<br>- New connection latency: ~2ms  ✅ 7x improvement<br>- Rule update time: ~500ms  ✅ 10x improvement</pre><p><strong>Production Recommendation</strong>: Alibaba Cloud ACK defaults to iptables. If Services exceed 500, recommend switching to IPVS.”</p><h3>Switching to IPVS Mode</h3><pre># 1. Check current mode<br>kubectl logs -n kube-system &lt;kube-proxy-pod&gt; | grep &quot;Using&quot;<br># Output: Using iptables Proxier<br>​<br># 2. Modify kube-proxy configuration<br>kubectl edit configmap kube-proxy -n kube-system<br>​<br># Modify:<br># mode: &quot;&quot;  →  mode: &quot;ipvs&quot;<br>​<br># 3. Restart kube-proxy<br>kubectl rollout restart daemonset kube-proxy -n kube-system<br>​<br># 4. Verify<br>ipvsadm -Ln | head -20<br># Should see virtual server list</pre><h3>Step Six: Deep Tuning</h3><h3>ALB Tuning</h3><pre># Ingress annotations<br>metadata:<br>  annotations:<br>    # 1. Connection optimization<br>    alb.ingress.kubernetes.io/backend-keepalive: &quot;true&quot;<br>    alb.ingress.kubernetes.io/backend-keepalive-timeout: &quot;90&quot;<br>    alb.ingress.kubernetes.io/connection-idle-timeout: &quot;60&quot;<br>​<br>    # 2. Health check optimization<br>    alb.ingress.kubernetes.io/healthcheck-interval-seconds: &quot;5&quot;  # Reduce to 5 seconds<br>    alb.ingress.kubernetes.io/healthy-threshold-count: &quot;2&quot;  # 2 successes = healthy<br>    alb.ingress.kubernetes.io/unhealthy-threshold-count: &quot;2&quot;  # 2 failures = unhealthy<br>​<br>    # 3. Timeout optimization<br>    alb.ingress.kubernetes.io/backend-timeout: &quot;30&quot;  # Backend timeout 30 seconds<br>​<br>    # 4. Traffic optimization<br>    alb.ingress.kubernetes.io/traffic-limit-qps: &quot;10000&quot;  # Limit QPS</pre><h3>Service Tuning</h3><pre>apiVersion: v1<br>kind: Service<br>spec:<br>  # 1. Session affinity (if needed)<br>  sessionAffinity: ClientIP<br>  sessionAffinityConfig:<br>    clientIP:<br>      timeoutSeconds: 10800  # 3 hours<br>​<br>  # 2. External traffic policy<br>  externalTrafficPolicy: Local  # Only forward traffic to Pods on this Node (reduce cross-Node forwarding)<br>​<br>  # 3. Internal traffic policy<br>  internalTrafficPolicy: Local  # Internal cluster calls also prefer this Node</pre><p><strong>externalTrafficPolicy Comparison</strong>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KVm4FTq6bNT8nQFqOlD0EA.png" /></figure><p><strong>Recommendation</strong>:</p><ul><li><strong>High performance requirements</strong>: Local (reduce network hops)</li><li><strong>Load balancing priority</strong>: Cluster (default)</li></ul><h3>Summary: The Wisdom of Dual-Layer Architecture</h3><p>Through in-depth conversations with Claude, I thoroughly understood the design philosophy of dual-layer load balancing:</p><h3>🎯 Core Principles</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WiGmTrrqPKBaDBx_Q5WrYg.png" /></figure><h3>📋 Key Takeaways</h3><p>✅ <strong>Service is not a process, it’s a configuration object</strong></p><p>✅ <strong>kube-proxy is a rule generator, iptables/IPVS is the executor</strong></p><p>✅ <strong>Dual-layer load balancing solves all single-layer solution problems</strong></p><p>✅ <strong>ALB handles cross-Node, iptables handles cross-Pod</strong></p><p>✅ <strong>Automatic failover: Node-level (ALB) + Pod-level (iptables)</strong></p><p>✅ <strong>Zero-downtime rolling updates: Readiness + graceful shutdown + iptables auto-update</strong></p><p>✅ <strong>Performance tuning: IPVS mode + externalTrafficPolicy</strong></p><h3>🔍 Visual Summary</h3><pre>User Request (HTTPS)<br>   ↓<br>【First Layer: Alibaba Cloud ALB】<br>   ├─ TLS Termination (decrypt)<br>   ├─ Layer 7 Routing (domain + path)<br>   ├─ Health Checks (Node-level)<br>   └─ Load Balancing (cross-Node)<br>   ↓<br>Node:31234 (NodePort)<br>   ↓<br>【Second Layer: iptables/IPVS】<br>   ├─ DNAT (port translation)<br>   ├─ Load Balancing (cross-Pod)<br>   ├─ Connection Tracking (CONNTRACK)<br>   └─ Auto-update (tracks Pod changes)<br>   ↓<br>Pod:8080 (container port)<br>   ↓<br>Application processing<br>   ↓<br>Response returns (SNAT, returns same path)</pre><h3>Next Steps: Production Environment Checklist</h3><p>After understanding the principles of dual-layer load balancing, you’ve mastered the essence of K8s networking. In the final article, I will share <strong>a complete pre-production checklist</strong> to ensure your application is foolproof.</p><h3>Appendix: Troubleshooting Command Reference</h3><pre># ========== View Service ==========<br>kubectl get service -n production<br>kubectl describe service demo-app -n production<br>kubectl get endpoints demo-app -n production  # View Pod IP list<br>​<br># ========== View iptables rules ==========<br># View Service rules<br>iptables -t nat -L KUBE-SERVICES -n | grep demo-app<br>​<br># View specific Service rule chain<br>iptables -t nat -L KUBE-SVC-XXXXX -n -v<br>​<br># ========== View IPVS rules ==========<br>ipvsadm -Ln | grep 10.0.100.50  # Service ClusterIP<br>​<br># ========== Test connectivity ==========<br># Test Service inside cluster<br>kubectl run test --rm -it --image=curlimages/curl -- curl http://demo-app:8080<br>​<br># Test iptables on Node<br>curl http://10.0.100.50:8080  # ClusterIP<br>​<br># View Pod network<br>kubectl exec -it &lt;pod-name&gt; -- ip addr<br>kubectl exec -it &lt;pod-name&gt; -- netstat -tunlp<br>​<br># ========== Packet capture analysis ==========<br># Capture on Node (view NodePort traffic)<br>tcpdump -i eth0 port 31234 -nn -A<br>​<br># Capture in Pod (view container traffic)<br>kubectl exec -it &lt;pod-name&gt; -- tcpdump -i eth0 port 8080 -nn<br>​<br># ========== View kube-proxy logs ==========<br>kubectl logs -n kube-system &lt;kube-proxy-pod&gt; --tail=100<br>​<br># ========== Performance analysis ==========<br># iptables rule count<br>iptables -t nat -L | wc -l<br>​<br># IPVS connection count<br>ipvsadm -Ln --stats<br>​<br># View Service count<br>kubectl get service --all-namespaces | wc -l</pre><p><strong>Author</strong>: [TinyTwo] <strong>Creation Tool</strong>: This article was completed with the assistance of Claude AI. All diagrams were created using Mermaid. <strong>Series</strong>: [4/5] — Next article: “Production Deployment Guide: AI-Assisted Production Environment Checklist”</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=efb1bb455db1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Traffic Management: IngressClass Three-Tier Architecture Explained]]></title>
            <link>https://medium.com/@guqung6/traffic-management-ingressclass-three-tier-architecture-explained-61827d8fb4af?source=rss-e1958981e79b------2</link>
            <guid isPermaLink="false">https://medium.com/p/61827d8fb4af</guid>
            <category><![CDATA[alibaba]]></category>
            <category><![CDATA[deployment]]></category>
            <category><![CDATA[k8s]]></category>
            <category><![CDATA[claude]]></category>
            <category><![CDATA[kubernetes]]></category>
            <dc:creator><![CDATA[TinyTwo]]></dc:creator>
            <pubDate>Mon, 02 Feb 2026 13:01:02 GMT</pubDate>
            <atom:updated>2026-02-02T13:01:02.682Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XuQuK_W6imNufcSU3YmLEw.jpeg" /></figure><h3>Introduction: From Chaos to Order</h3><p>In the previous article, we successfully deployed our application, but it was only accessible within the cluster. To allow external users to access it, we need <strong>Ingress</strong>.</p><p>When I asked Claude “How to configure Ingress”, he gave me an unexpected answer:</p><blockquote><strong><em>Claude</em></strong><em>: “Wait, are you using Alibaba Cloud ACK? What’s your K8s version?”</em></blockquote><blockquote><strong><em>Me</em></strong><em>: “Yes, ACK version 1.24.”</em></blockquote><blockquote><strong><em>Claude</em></strong><em>: “Then you need to understand an important change: </em><strong><em>Ingress configuration has been upgraded in K8s 1.18+</em></strong><em>.</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wApwKdDBZHGLo-qkRj5STQ.png" /></figure><p>This conversation made me realize: <strong>following outdated tutorials is the biggest pitfall</strong>. Fortunately, Claude helped me avoid this trap.</p><h3>Step 1: Understanding the Three-Tier Architecture</h3><h3>My Confusion</h3><blockquote><strong><em>Me</em></strong><em>: “Why do we need three tiers? Can’t we just write ALB configuration directly in Ingress?”</em></blockquote><blockquote><strong><em>Claude</em></strong><em>: “Imagine this scenario: Your company has 3 teams sharing one K8s cluster:</em></blockquote><blockquote><strong><em>Old Method</em></strong><em> (Annotation Hell):</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Dn95EFDA1R09zO4gKBgzeA.png" /></figure><blockquote><strong><em>Problems</em></strong><em>:</em></blockquote><ul><li>❌ Every team has to copy-paste configuration (configuration drift risk)</li><li>❌ Modifying ALB config requires changing all Ingresses</li><li>❌ New team members don’t know which annotations to write</li><li>❌ Multiple people modifying the same ALB easily causes conflicts</li></ul><blockquote><strong><em>New Method</em></strong><em> (Three-tier Architecture):</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HxydI0tXFmEQVSyGyXcmeA.png" /></figure><blockquote><strong><em>Advantages</em></strong><em>:</em></blockquote><ul><li>✅ Configuration reuse (100 Ingresses share 1 AlbConfig)</li><li>✅ Separation of duties (platform manages ALB, business manages routes)</li><li>✅ Unified changes (modify AlbConfig only, no need to change Ingress)</li><li>✅ Multi-tenant isolation (different teams use different IngressClasses)”</li></ul><p>I drew a comparison diagram:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*t1LdT8x6ncNXl8gnieWyNA.png" /></figure><h3>Step 2: Three-Tier Architecture Explained</h3><h3>Architecture Overview</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*52jU08SxgcEc9jrUE4oFBA.png" /></figure><h3>First Tier: AlbConfig (ALB Configuration Object)</h3><p><strong>Purpose</strong>: Define the physical attributes of Alibaba Cloud ALB (specifications, network, billing method, etc.)</p><pre>apiVersion: alibabacloud.com/v1<br>kind: AlbConfig<br>metadata:<br>  name: default-alb<br>  namespace: kube-system  # Must be in kube-system<br>spec:<br>  config:<br>    # ALB instance configuration<br>    name: k8s-prod-alb  # ALB instance name<br>    addressType: Internet  # Internet (public) or Intranet (private)<br>    zoneMappings:  # Availability zone configuration (at least 2 for high availability)<br>    - vSwitchId: vsw-bp1a2b3c4d5e6f7g8  # VSwitch in zone A<br>    - vSwitchId: vsw-bp9h8i7j6k5l4m3n2  # VSwitch in zone B<br>​<br>    # Listener configuration<br>    listeners:<br>    - port: 80<br>      protocol: HTTP<br>      description: &quot;HTTP Listener&quot;<br>​<br>    - port: 443<br>      protocol: HTTPS<br>      description: &quot;HTTPS Listener&quot;<br>      certificates:<br>      - CertificateId: &quot;cert-123456&quot;  # Alibaba Cloud certificate ID<br>        IsDefault: true<br>​<br>    # Access control (optional)<br>    accessLogConfig:<br>      logProject: &quot;k8s-alb-logs&quot;<br>      logStore: &quot;alb-access-log&quot;<br>​<br>    # Other configurations<br>    addressAllocatedMode: Dynamic  # Dynamic (dynamic IP) or Fixed (fixed IP)<br>    deletionProtectionEnabled: true  # Deletion protection<br>    edition: Standard  # Standard or StandardWithWaf (WAF version)</pre><p><strong>Key Configuration Explanation</strong>:</p><p>ConfigurationDescriptionProduction RecommendationaddressTypePublic/Private networkUse Internet for public services, Intranet for internalzoneMappingsAvailability zone config<strong>At least 2 zones</strong> (high availability)editionALB versionStandard is sufficient, choose StandardWithWaf if WAF neededdeletionProtectionEnabledDeletion protection<strong>Must enable</strong> in production</p><h3>Second Tier: IngressClass (Logical Grouping)</h3><p><strong>Purpose</strong>: Associate AlbConfig and Ingress Controller, enable multi-tenant isolation</p><pre>apiVersion: networking.k8s.io/v1<br>kind: IngressClass<br>metadata:<br>  name: alb  # Ingress will reference this name<br>spec:<br>  controller: ingress.k8s.alibabacloud/alb-controller  # Specify Controller<br>​<br>  parameters:<br>    apiGroup: alibabacloud.com<br>    kind: AlbConfig<br>    name: default-alb  # Associate with the above AlbConfig</pre><p><strong>Multi-tenant Scenario</strong>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Wv6imnbMv70ZZVGlCvC8KA.png" /></figure><h3>Third Tier: Ingress (Routing Rules)</h3><p><strong>Purpose</strong>: Define routing rules from domain/path to Service</p><pre>apiVersion: networking.k8s.io/v1<br>kind: Ingress<br>metadata:<br>  name: demo-ingress<br>  namespace: production<br>spec:<br>  ingressClassName: alb  # Reference IngressClass<br>​<br>  # Routing rules<br>  rules:<br>  # Rule 1: API domain<br>  - host: api.example.com<br>    http:<br>      paths:<br>      - path: /v1/users<br>        pathType: Prefix<br>        backend:<br>          service:<br>            name: user-service<br>            port:<br>              number: 8080<br>​<br>      - path: /v1/orders<br>        pathType: Prefix<br>        backend:<br>          service:<br>            name: order-service<br>            port:<br>              number: 8080<br>​<br>  # Rule 2: Web domain<br>  - host: www.example.com<br>    http:<br>      paths:<br>      - path: /<br>        pathType: Prefix<br>        backend:<br>          service:<br>            name: web-frontend<br>            port:<br>              number: 80<br>​<br>  # HTTPS configuration<br>  tls:<br>  - hosts:<br>    - api.example.com<br>    - www.example.com<br>    secretName: example-tls  # Secret containing certificate</pre><h3>Step 3: Complete Deployment Practice</h3><p>My complete conversation with Claude:</p><h3>Conversation 1: Preparation</h3><blockquote><strong><em>Me</em></strong><em>: “On Alibaba Cloud ACK, what preparation do we need?”</em></blockquote><blockquote><strong><em>Claude</em></strong><em>: “1. </em><strong><em>Confirm ALB Ingress Controller is installed</em></strong></blockquote><blockquote><em>kubectl get deployment -n kube-system | grep alb<br># Should see alb-ingress-controller</em></blockquote><ul><li><strong>Prepare Alibaba Cloud resources</strong>:</li><li>VPC VSwitch IDs (at least 2 zones)</li><li>SSL certificate ID (if using HTTPS)</li><li>Log project (if enabling access logs)</li><li><strong>Prepare K8s resources</strong>:</li><li>TLS Secret (certificate)</li><li>Backend Service already deployed</li><li><strong>Permission confirmation</strong>:</li><li>Controller’s ServiceAccount has permission to create ALB”</li></ul><h3>Conversation 2: Creating TLS Secret</h3><blockquote><strong><em>Me</em></strong><em>: “How to import certificate into K8s?”</em></blockquote><blockquote><strong><em>Claude</em></strong><em>: “If you have certificate files:</em></blockquote><blockquote><em>kubectl create secret tls example-tls \<br> — cert=cert.pem \<br> — key=key.pem \<br>-n production</em></blockquote><blockquote><em>If using Alibaba Cloud Certificate Service, directly reference the certificate ID in AlbConfig’s listeners.”</em></blockquote><h3>Conversation 3: Deployment Sequence</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IJJnY7mNQeb5YZCe1EKRpA.png" /></figure><h3>Complete Deployment Script</h3><pre># ========== Step 1: Create AlbConfig ==========<br>cat &lt;&lt;EOF | kubectl apply -f -<br>apiVersion: alibabacloud.com/v1<br>kind: AlbConfig<br>metadata:<br>  name: default-alb<br>  namespace: kube-system<br>spec:<br>  config:<br>    name: k8s-prod-alb<br>    addressType: Internet<br>    zoneMappings:<br>    - vSwitchId: vsw-bp1xxxxx  # Replace with your VSwitch ID<br>    - vSwitchId: vsw-bp2xxxxx<br>    listeners:<br>    - port: 443<br>      protocol: HTTPS<br>      certificates:<br>      - CertificateId: &quot;cert-xxxxx&quot;  # Replace with your certificate ID<br>        IsDefault: true<br>    - port: 80<br>      protocol: HTTP<br>    deletionProtectionEnabled: true<br>    edition: Standard<br>EOF<br>​<br># Wait for ALB creation (about 2-3 minutes)<br>kubectl get albconfig -n kube-system default-alb -w<br>​<br># ========== Step 2: Create IngressClass ==========<br>cat &lt;&lt;EOF | kubectl apply -f -<br>apiVersion: networking.k8s.io/v1<br>kind: IngressClass<br>metadata:<br>  name: alb<br>spec:<br>  controller: ingress.k8s.alibabacloud/alb-controller<br>  parameters:<br>    apiGroup: alibabacloud.com<br>    kind: AlbConfig<br>    name: default-alb<br>EOF<br>​<br># ========== Step 3: Create TLS Secret (if needed) ==========<br>kubectl create secret tls example-tls \<br>  --cert=cert.pem \<br>  --key=key.pem \<br>  -n production<br>​<br># ========== Step 4: Create Ingress ==========<br>cat &lt;&lt;EOF | kubectl apply -f -<br>apiVersion: networking.k8s.io/v1<br>kind: Ingress<br>metadata:<br>  name: demo-ingress<br>  namespace: production<br>  annotations:<br>    # Health check configuration<br>    alb.ingress.kubernetes.io/healthcheck-enabled: &quot;true&quot;<br>    alb.ingress.kubernetes.io/healthcheck-path: &quot;/healthz&quot;<br>    alb.ingress.kubernetes.io/healthcheck-protocol: &quot;HTTP&quot;<br>    alb.ingress.kubernetes.io/healthcheck-interval-seconds: &quot;5&quot;<br>    alb.ingress.kubernetes.io/healthcheck-timeout-seconds: &quot;3&quot;<br>​<br>    # Force HTTPS<br>    alb.ingress.kubernetes.io/ssl-redirect: &quot;true&quot;<br>​<br>spec:<br>  ingressClassName: alb<br>  rules:<br>  - host: api.example.com<br>    http:<br>      paths:<br>      - path: /<br>        pathType: Prefix<br>        backend:<br>          service:<br>            name: demo-app  # Reference previously created Service<br>            port:<br>              number: 8080<br>  tls:<br>  - hosts:<br>    - api.example.com<br>    secretName: example-tls<br>EOF<br>​<br># ========== Step 5: Check Ingress status ==========<br>kubectl get ingress -n production demo-ingress<br>​<br># ========== Step 6: Get ALB address ==========<br>kubectl get ingress -n production demo-ingress -o jsonpath=&#39;{.status.loadBalancer.ingress[0].hostname}&#39;<br># Output like: alb-xxxxx.cn-hangzhou.alb.aliyuncs.com<br>​<br># ========== Step 7: Configure DNS ==========<br># Configure at your DNS provider:<br># api.example.com CNAME alb-xxxxx.cn-hangzhou.alb.aliyuncs.com</pre><h3>Step 4: Advanced Configuration</h3><h3>HTTPS Redirect</h3><blockquote><strong><em>Me</em></strong><em>: “How to force users to use HTTPS?”</em></blockquote><blockquote><strong><em>Claude</em></strong><em>: “Add annotation in Ingress:</em></blockquote><blockquote><em>metadata:<br>annotations:<br> alb.ingress.kubernetes.io/ssl-redirect: \”true\”</em></blockquote><blockquote><em>Effect: Accessing </em><a href="http://api.example.com/"><em>http://api.example.com</em></a><em> automatically redirects to </em><a href="https://api.example.com/"><em>https://api.example.com</em></a><em>&quot;</em></blockquote><h3>Custom Health Check</h3><pre>metadata:<br>  annotations:<br>    # Health check configuration<br>    alb.ingress.kubernetes.io/healthcheck-enabled: &quot;true&quot;<br>    alb.ingress.kubernetes.io/healthcheck-path: &quot;/healthz&quot;<br>    alb.ingress.kubernetes.io/healthcheck-protocol: &quot;HTTP&quot;<br>    alb.ingress.kubernetes.io/healthcheck-method: &quot;GET&quot;<br>    alb.ingress.kubernetes.io/healthcheck-interval-seconds: &quot;5&quot;<br>    alb.ingress.kubernetes.io/healthcheck-timeout-seconds: &quot;3&quot;<br>    alb.ingress.kubernetes.io/healthcheck-httpcode: &quot;http_2xx&quot;<br>    alb.ingress.kubernetes.io/healthy-threshold-count: &quot;2&quot;<br>    alb.ingress.kubernetes.io/unhealthy-threshold-count: &quot;3&quot;</pre><p><strong>Parameter Description</strong>:</p><ul><li>healthcheck-path: Health check path (should match Pod&#39;s Readiness)</li><li>healthcheck-interval-seconds: Check interval</li><li>healthy-threshold-count: 2 consecutive successes mark as healthy</li><li>unhealthy-threshold-count: 3 consecutive failures mark as unhealthy</li></ul><h3>Session Persistence</h3><pre>metadata:<br>  annotations:<br>    alb.ingress.kubernetes.io/session-sticky: &quot;true&quot;<br>    alb.ingress.kubernetes.io/session-sticky-type: &quot;Insert&quot;  # Insert or Server<br>    alb.ingress.kubernetes.io/session-sticky-timeout: &quot;3600&quot;  # 1 hour</pre><h3>CORS Configuration</h3><pre>metadata:<br>  annotations:<br>    alb.ingress.kubernetes.io/cors-allow-origin: &quot;https://www.example.com&quot;<br>    alb.ingress.kubernetes.io/cors-allow-methods: &quot;GET,POST,PUT,DELETE&quot;<br>    alb.ingress.kubernetes.io/cors-allow-headers: &quot;Content-Type,Authorization&quot;<br>    alb.ingress.kubernetes.io/cors-max-age: &quot;3600&quot;</pre><h3>Step 5: Migration Guide (Old Method → New Method)</h3><h3>Old Method Example</h3><pre># ❌ Old method: All configuration in annotations<br>apiVersion: networking.k8s.io/v1<br>kind: Ingress<br>metadata:<br>  name: old-ingress<br>  annotations:<br>    kubernetes.io/ingress.class: alb<br>    alb.ingress.kubernetes.io/loadbalancer-id: lb-xxxxx<br>    alb.ingress.kubernetes.io/vswitch-ids: vsw-001,vsw-002<br>    alb.ingress.kubernetes.io/address-type: internet<br>    alb.ingress.kubernetes.io/healthcheck-enabled: &quot;true&quot;<br>    alb.ingress.kubernetes.io/healthcheck-path: &quot;/healthz&quot;<br>    # ...20+ more annotations<br>spec:<br>  rules:<br>  - host: api.example.com<br>    http:<br>      paths:<br>      - path: /<br>        backend:<br>          serviceName: demo-app<br>          servicePort: 8080</pre><h3>New Method Conversion</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vDdTU11EGNX21kZ97rfp_A.png" /></figure><p><strong>Migration Steps</strong>:</p><ol><li><strong>Extract ALB configuration</strong> → Create AlbConfig</li><li><strong>Create IngressClass</strong> → Associate AlbConfig</li><li><strong>Simplify Ingress</strong> → Remove annotations, add ingressClassName</li><li><strong>Gradual switch</strong> → Coexist old and new Ingress for testing</li><li><strong>Clean up old resources</strong> → Delete old Ingress</li></ol><h3>Step 6: Verification and Troubleshooting</h3><h3>Verification Checklist</h3><pre># ✅ 1. AlbConfig status<br>kubectl get albconfig -n kube-system default-alb -o yaml<br># Check status.loadBalancer.id, confirm ALB is created<br>​<br># ✅ 2. IngressClass exists<br>kubectl get ingressclass alb<br>​<br># ✅ 3. Ingress status<br>kubectl describe ingress -n production demo-ingress<br># Check Events, confirm no errors<br>​<br># ✅ 4. Ingress address<br>kubectl get ingress -n production demo-ingress<br># ADDRESS column should show ALB domain<br>​<br># ✅ 5. Test access<br>curl -k https://api.example.com/healthz<br># Should return 200 OK<br>​<br># ✅ 6. Check ALB backend server group<br># Login to Alibaba Cloud console → Application Load Balancer ALB → View backend server group<br># Should see K8s Nodes as backend servers</pre><h3>Common Issues</h3><h4>Issue 1: Ingress has no ADDRESS</h4><blockquote><strong><em>Me</em></strong><em>: “Ingress is created, but ADDRESS column is always empty?”</em></blockquote><blockquote><strong><em>Claude</em></strong><em>: “Troubleshooting steps:</em></blockquote><blockquote><em># 1. Check Controller logs<br>kubectl logs -n kube-system deployment/alb-ingress-controller<br>​<br># 2. Check AlbConfig status<br>kubectl get albconfig -n kube-system default-alb -o yaml | grep -A 5 status<br>​<br># 3. Check IngressClass name<br>kubectl get ingressclass<br># Confirm Ingress’s ingressClassName matches actual IngressClass name</em></blockquote><blockquote><strong><em>Common causes</em></strong><em>:</em></blockquote><ul><li>Incorrect vSwitchId in AlbConfig</li><li>Insufficient Controller permissions (cannot create ALB)</li><li>IngressClass name misspelled”</li></ul><h4>Issue 2: HTTPS not working</h4><blockquote><strong><em>Me</em></strong><em>: “Accessing HTTPS returns certificate error?”</em></blockquote><blockquote><strong><em>Claude</em></strong><em>: “Check:</em></blockquote><ul><li><strong>Is Secret in the same namespace</strong></li></ul><pre>kubectl get secret -n production example-tls</pre><ul><li><strong>Is certificate content correct</strong></li></ul><pre>kubectl get secret example-tls -o jsonpath=&#39;{.data.tls\.crt}&#39; | base64 -d | openssl x509 -text -noout<br># Check Subject and SubjectAltName, confirm domain matches</pre><ul><li><strong>Is certificate ID correct in AlbConfig</strong> (if using Alibaba Cloud certificate)”</li></ul><h4>Issue 3: 503 Service Unavailable</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BCrglTcRFfBJRRA0V3gE7Q.png" /></figure><h3>Summary: Configuration Checklist</h3><h3>📋 AlbConfig Configuration</h3><ul><li>namespace must be kube-system</li><li>At least 2 vSwitchIds (cross-zone high availability)</li><li>addressType choose Internet (public) or Intranet (private)</li><li>deletionProtectionEnabled set to true in production</li><li>Certificate ID correct (if using HTTPS)</li></ul><h3>📋 IngressClass Configuration</h3><ul><li>controller value is ingress.k8s.alibabacloud/alb-controller</li><li>parameters correctly associate AlbConfig</li><li>Name is easy to identify (e.g., alb, alb-prod, alb-test)</li></ul><h3>📋 Ingress Configuration</h3><ul><li>ingressClassName matches IngressClass name</li><li>host configured correctly (full domain name)</li><li>path matches Service path</li><li>Service exists and has Endpoints</li><li>TLS Secret in same namespace</li><li>healthcheck path matches Pod’s Readiness</li></ul><h3>📋 DNS Configuration</h3><ul><li>Domain CNAME to ALB address</li><li>DNS resolution effective (verify with dig/nslookup)</li><li>Certificate domain matches configured host</li></ul><h3>Next Step: Deep Dive into Load Balancing Principles</h3><p>Now you’ve configured Ingress and traffic can enter the cluster from outside. But you might be curious:</p><ul><li><strong>Why do we need both ALB and Service for two-tier load balancing?</strong></li><li><strong>How do packets travel from ALB to Pod?</strong></li><li><strong>Who is doing Service’s load balancing?</strong></li></ul><p>The next article will deep dive into <strong>K8s dual-tier load balancing principles</strong> and solve these mysteries.</p><p><strong>Key Takeaways from this Article</strong>:</p><p>✅ IngressClass three-tier architecture: AlbConfig → IngressClass → Ingress</p><p>✅ AlbConfig defines ALB physical attributes, managed by platform team</p><p>✅ IngressClass enables multi-tenant isolation and configuration reuse</p><p>✅ Ingress only writes routing rules, configuration is concise</p><p>✅ Old method (annotation) is outdated, new projects must use IngressClass</p><p>✅ Health check configuration must match Pod’s Readiness</p><h3>Appendix: Complete Configuration Template</h3><pre># ========== AlbConfig ==========<br>apiVersion: alibabacloud.com/v1<br>kind: AlbConfig<br>metadata:<br>  name: default-alb<br>  namespace: kube-system<br>spec:<br>  config:<br>    name: k8s-prod-alb<br>    addressType: Internet<br>    zoneMappings:<br>    - vSwitchId: vsw-xxxxx<br>    - vSwitchId: vsw-yyyyy<br>    listeners:<br>    - port: 443<br>      protocol: HTTPS<br>      certificates:<br>      - CertificateId: &quot;cert-xxxxx&quot;<br>        IsDefault: true<br>    - port: 80<br>      protocol: HTTP<br>    deletionProtectionEnabled: true<br>    edition: Standard<br>​<br>---<br># ========== IngressClass ==========<br>apiVersion: networking.k8s.io/v1<br>kind: IngressClass<br>metadata:<br>  name: alb<br>spec:<br>  controller: ingress.k8s.alibabacloud/alb-controller<br>  parameters:<br>    apiGroup: alibabacloud.com<br>    kind: AlbConfig<br>    name: default-alb<br>​<br>---<br># ========== Ingress ==========<br>apiVersion: networking.k8s.io/v1<br>kind: Ingress<br>metadata:<br>  name: demo-ingress<br>  namespace: production<br>  annotations:<br>    alb.ingress.kubernetes.io/healthcheck-enabled: &quot;true&quot;<br>    alb.ingress.kubernetes.io/healthcheck-path: &quot;/healthz&quot;<br>    alb.ingress.kubernetes.io/ssl-redirect: &quot;true&quot;<br>spec:<br>  ingressClassName: alb<br>  rules:<br>  - host: api.example.com<br>    http:<br>      paths:<br>      - path: /<br>        pathType: Prefix<br>        backend:<br>          service:<br>            name: demo-app<br>            port:<br>              number: 8080<br>  tls:<br>  - hosts:<br>    - api.example.com<br>    secretName: example-tls</pre><p><strong>Author</strong>: [TinyTwo] <strong>Creation Tool</strong>: This article was completed with the assistance of Claude AI, all architecture diagrams drawn with Mermaid <strong>Article Series</strong>: [3/5] — Next: “Architecture Principles Revealed: Deep Dive into Dual-Tier Load Balancing”</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=61827d8fb4af" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[K8s Practical Journey: Deploying Your First App with Claude]]></title>
            <link>https://medium.com/@guqung6/k8s-practical-journey-deploying-your-first-app-with-claude-c5491310a038?source=rss-e1958981e79b------2</link>
            <guid isPermaLink="false">https://medium.com/p/c5491310a038</guid>
            <category><![CDATA[deployment]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[vibe-coding]]></category>
            <category><![CDATA[claude]]></category>
            <category><![CDATA[k8s]]></category>
            <dc:creator><![CDATA[TinyTwo]]></dc:creator>
            <pubDate>Tue, 27 Jan 2026 02:25:54 GMT</pubDate>
            <atom:updated>2026-01-27T02:25:54.200Z</atom:updated>
            <content:encoded><![CDATA[<h3>deK8s Practical Journey: Deploying Your First App with Claude</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7-eq4-j_dYUTIuwS2lpbyw.jpeg" /></figure><h3>Introduction: From Concepts to Practice</h3><p>In the previous article, we understood the four core objects of K8s. However, there&#39;s still a gap between understanding concepts and hands-on deployment. This article will <strong>completely document my deployment dialogue with Claude</strong>, demonstrating how to deploy a production-grade application from scratch.</p><p><strong>Deployment Goals</strong>:</p><ul><li>✅ A web application (3 replicas for high availability)</li><li>✅ Health checks (automatic restart of failed containers)</li><li>✅ Resource limits (prevent single container from exhausting resources)</li><li>✅ Environment configuration (ConfigMap management)</li><li>✅ Internal access (Service load balancing)</li></ul><h3>Step 1: Define Requirements</h3><h3>My Conversation with Claude</h3><blockquote><strong><em>Me</em></strong><em>: &quot;Claude, I want to deploy a web application on Alibaba Cloud ACK. What do I need to prepare?</em></blockquote><blockquote><strong><em>Claude</em></strong><em>: &quot;Let me help you organize the deployment checklist:</em></blockquote><blockquote><strong><em>Required K8s Objects</em></strong><em>:</em></blockquote><ul><li><strong>ConfigMap</strong> - Store configuration (environment variables, config files)</li><li><strong>Deployment</strong> - Define how the application runs</li><li><strong>Service</strong> - Provide internal access endpoint</li></ul><blockquote><strong><em>Key Configuration Items</em></strong><em>:</em></blockquote><ul><li>Image address (recommended to use Alibaba Cloud Container Registry)</li><li>Number of replicas (at least 2 for production environment)</li><li>Resource limits (CPU/memory requests and limits)</li><li>Health checks (Liveness and Readiness probes)</li><li>Port mapping</li></ul><blockquote><strong><em>Deployment Order</em></strong><em>: ConfigMap → Deployment → Service&quot;</em></blockquote><p>I had an epiphany: <strong>Deployment is not a single YAML file, but a combination of multiple objects</strong>.</p><h3>Step 2: Configuration Management (ConfigMap)</h3><h3>My Confusion</h3><blockquote><strong><em>Me</em></strong><em>: &quot;Why do we need ConfigMap? Can&#39;t we just write the configuration directly in the Deployment?&quot;</em></blockquote><blockquote><strong><em>Claude</em></strong><em>: &quot;</em><strong><em>Separation of configuration and code</em></strong><em> is a best practice:</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*r0BTWfgx6cO1rELy1d1mEw.png" /></figure><blockquote><strong><em>Real Scenario</em></strong><em>: Your application uses different database addresses in dev/test/prod environments. With ConfigMap, you can achieve </em><strong><em>same Deployment configuration, inject different ConfigMaps based on environment</em></strong><em>.&quot;</em></blockquote><h3>ConfigMap Configuration</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*B1jYekUD5dWimesyvbwKyg.png" /></figure><p><strong>Configuration Notes</strong>:</p><ul><li>metadata.namespace: Recommend using different namespaces for different environments (production/staging/development)</li><li>data: Key-Value format, supports single-line strings or multi-line text (| symbol)</li><li><strong>Naming Convention</strong>: {app-name}-config, easy to identify</li></ul><h3>Deploy ConfigMap</h3><pre>kubectl apply -f configmap.yaml<br>kubectl get configmap -n production</pre><h3>Step 3: Application Orchestration (Deployment)</h3><p>This is the most complex part, and I had multiple rounds of dialogue with Claude.</p><h3>Dialogue 1: Basic Configuration</h3><blockquote><strong><em>Me</em></strong><em>: &quot;What is the basic structure of a Deployment?&quot;</em></blockquote><blockquote><strong><em>Claude</em></strong><em>: &quot;Deployment uses a </em><strong><em>nested structure</em></strong><em>:</em></blockquote><blockquote><em>Deployment</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ue5c4EslZnxWFUUQRdKnig.png" /></figure><blockquote><strong><em>Core Concept</em></strong><em>: Deployment defines &#39;what kind of Pods I want&#39;, Controller Manager is responsible for &#39;creating and maintaining these Pods&#39;.&quot;</em></blockquote><h3>Dialogue 2: Health Check Configuration</h3><blockquote><strong><em>Me</em></strong><em>: &quot;What&#39;s the difference between Liveness and Readiness probes? Do both need to be configured?&quot;</em></blockquote><blockquote><strong><em>Claude</em></strong><em>: &quot;This is the most confusing point, see the comparison:</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Tvt2r3g7jiA8prxloCg1OA.png" /></figure><blockquote><strong><em>Production Recommendation</em></strong><em>: Configure both!</em></blockquote><ul><li>Liveness ensures container won&#39;t be stuck forever</li><li>Readiness ensures users won&#39;t access unready Pods&quot;</li></ul><p>I drew a sequence diagram to understand:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/743/1*x7xF6nO-ItiJ9q2TevEB7Q.png" /></figure><h3>Dialogue 3: Resource Limits</h3><blockquote><strong><em>Me</em></strong><em>: &quot;What&#39;s the difference between requests and limits?&quot;</em></blockquote><blockquote><strong><em>Claude</em></strong><em>: &quot;This determines Pod </em><strong><em>scheduling and eviction</em></strong><em>:</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uSruY_1UG25UwSTjU8br-Q.png" /></figure><h3>Complete Deployment Configuration</h3><pre>apiVersion: apps/v1<br>kind: Deployment<br>metadata:<br>  name: demo-app<br>  namespace: production<br>  labels:<br>    app: demo-app<br>    version: v1.0.0<br>spec:<br>  replicas: 3  # High availability: at least 3 replicas<br><br>  # Rolling update strategy<br>  strategy:<br>    type: RollingUpdate<br>    rollingUpdate:<br>      maxUnavailable: 1  # At most 1 Pod unavailable<br>      maxSurge: 1        # At most 1 extra Pod created<br><br>  # Selector: associate Pods<br>  selector:<br>    matchLabels:<br>      app: demo-app<br><br>  # Pod template<br>  template:<br>    metadata:<br>      labels:<br>        app: demo-app<br>        version: v1.0.0<br>    spec:<br>      containers:<br>      - name: app<br>        image: registry.cn-hangzhou.aliyuncs.com/demo/app:1.0.0<br><br>        # Ports<br>        ports:<br>        - name: http<br>          containerPort: 8080<br>          protocol: TCP<br><br>        # Environment variables: inject from ConfigMap<br>        envFrom:<br>        - configMapRef:<br>            name: demo-app-config<br><br>        # Additional environment variables (can override ConfigMap)<br>        env:<br>        - name: POD_NAME<br>          valueFrom:<br>            fieldRef:<br>              fieldPath: metadata.name<br>        - name: POD_IP<br>          valueFrom:<br>            fieldRef:<br>              fieldPath: status.podIP<br><br>        # Resource limits<br>        resources:<br>          requests:<br>            cpu: 200m       # 0.2 cores<br>            memory: 256Mi   # 256MB<br>          limits:<br>            cpu: 500m       # 0.5 cores<br>            memory: 512Mi   # 512MB<br><br>        # Liveness probe: detect if container is alive<br>        livenessProbe:<br>          httpGet:<br>            path: /healthz<br>            port: 8080<br>            scheme: HTTP<br>          initialDelaySeconds: 30  # First check after 30 seconds (give app time to start)<br>          periodSeconds: 10        # Check every 10 seconds<br>          timeoutSeconds: 5        # 5 second timeout<br>          successThreshold: 1      # 1 consecutive success considered healthy<br>          failureThreshold: 3      # Restart after 3 consecutive failures<br><br>        # Readiness probe: detect if container is ready<br>        readinessProbe:<br>          httpGet:<br>            path: /ready<br>            port: 8080<br>          initialDelaySeconds: 10  # First check after 10 seconds<br>          periodSeconds: 5         # Check every 5 seconds<br>          timeoutSeconds: 3<br>          successThreshold: 1<br>          failureThreshold: 3      # Remove from Service after 3 consecutive failures<br><br>        # Startup probe (optional): for slow-starting applications<br>        startupProbe:<br>          httpGet:<br>            path: /healthz<br>            port: 8080<br>          initialDelaySeconds: 0<br>          periodSeconds: 10<br>          failureThreshold: 30     # Wait up to 300 seconds (30*10)<br><br>        # Mount ConfigMap files<br>        volumeMounts:<br>        - name: config-volume<br>          mountPath: /etc/config<br><br>      # Define Volume<br>      volumes:<br>      - name: config-volume<br>        configMap:<br>          name: demo-app-config<br>          items:<br>          - key: app.conf<br>            path: app.conf<br><br>      # Other configurations<br>      restartPolicy: Always  # Always restart when container exits<br>      terminationGracePeriodSeconds: 30  # Graceful shutdown wait time</pre><h3>Rolling Update Process</h3><p>I asked Claude: “How does rolling update work when updating the image?”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TasSgxKcH_EwX040Wb3KHA.png" /></figure><p><strong>Key Parameters</strong>:</p><ul><li>maxUnavailable: 1 → Ensure at least 2 Pods available (3-1=2)</li><li>maxSurge: 1 → At most 4 Pods during update (3+1=4)</li><li>Readiness probe → Ensure new Pod is ready before deleting old Pod</li></ul><h3>Step 4: Service Exposure (Service)</h3><h3>Dialogue: Service Type Selection</h3><blockquote><strong><em>Me</em></strong><em>: “Service has ClusterIP, NodePort, LoadBalancer. Which should I choose?”</em></blockquote><blockquote><strong><em>Claude</em></strong><em>: “</em><strong><em>Recommended approach</em></strong><em> on Alibaba Cloud ACK:</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LFofLJzrKL2Be1gsEQ1lpg.png" /></figure><h3>Service Configuration</h3><pre>apiVersion: v1<br>kind: Service<br>metadata:<br>  name: demo-app<br>  namespace: production<br>  labels:<br>    app: demo-app<br>spec:<br>  type: ClusterIP  # Internal cluster access<br>​<br>  # Selector: match Pods with label app=demo-app<br>  selector:<br>    app: demo-app<br>​<br>  # Port mapping<br>  ports:<br>  - name: http<br>    protocol: TCP<br>    port: 8080        # Service port (other services use this port to access)<br>    targetPort: 8080  # Pod port (port container actually listens on)<br>​<br>  # Session affinity (optional)<br>  sessionAffinity: ClientIP  # Same client IP routes to same Pod<br>  sessionAffinityConfig:<br>    clientIP:<br>      timeoutSeconds: 10800  # 3 hours</pre><h3>Service Working Principle</h3><p>I asked: “How does Service find Pods? What happens when Pod IP changes?”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NHK14bIrn9R_iGbKacNYUw.png" /></figure><p><strong>Workflow</strong>:</p><ol><li><strong>Endpoints Controller</strong> continuously monitors Pod changes</li><li>Based on Service’s selector, automatically updates <strong>Endpoints object</strong> (Pod IP list)</li><li><strong>kube-proxy</strong> monitors Endpoints changes, updates iptables rules</li><li>When traffic arrives, iptables performs DNAT to specific Pod based on rules</li></ol><p><strong>Answer</strong>: After Pod IP changes, Endpoints automatically updates, kube-proxy automatically updates iptables rules, <strong>transparent to users</strong>!</p><h3>Step 5: Deployment and Verification</h3><h3>Deployment Commands</h3><pre># 1. Create namespace (if it doesn&#39;t exist)<br>kubectl create namespace production<br>​<br># 2. Deploy in order<br>kubectl apply -f configmap.yaml<br>kubectl apply -f deployment.yaml<br>kubectl apply -f service.yaml<br>​<br># Or deploy all at once (if in the same file)<br>kubectl apply -f demo-app.yaml</pre><h3>Verification Checklist</h3><p>I asked Claude to give me a verification checklist:</p><pre># ✅ Check 1: Is ConfigMap created<br>kubectl get configmap demo-app-config -n production<br>kubectl describe configmap demo-app-config -n production<br>​<br># ✅ Check 2: Is Deployment normal<br>kubectl get deployment demo-app -n production<br># Expected output: READY 3/3<br>​<br># ✅ Check 3: Are Pods running<br>kubectl get pods -n production -l app=demo-app<br># Expected output: 3 Pods, all in Running status<br>​<br># ✅ Check 4: Pod details (did health checks pass)<br>kubectl describe pod &lt;pod-name&gt; -n production<br># Check Events section, confirm Liveness/Readiness both successful<br>​<br># ✅ Check 5: Is Service created<br>kubectl get service demo-app -n production<br># Note the ClusterIP<br>​<br># ✅ Check 6: Does Endpoints have Pods<br>kubectl get endpoints demo-app -n production<br># Expected output: 3 Pod IPs<br>​<br># ✅ Check 7: Test Service connectivity (from inside cluster)<br>kubectl run test-pod --rm -it --image=curlimages/curl -- sh<br>curl http://demo-app.production.svc.cluster.local:8080/healthz<br># Expected output: 200 OK</pre><h3>Common Issue Troubleshooting</h3><p>I encountered issues during deployment, my troubleshooting dialogue with Claude:</p><h4>Issue 1: Pod Stuck in Pending</h4><blockquote><strong><em>Me</em></strong><em>: “Pod status is stuck in Pending, how to troubleshoot?”</em></blockquote><blockquote><strong><em>Claude</em></strong><em>: “Execute </em><em>kubectl describe pod &lt;pod-name&gt;, look at Events section:</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jVFFcSaOJvrpI-ms4JsSFQ.png" /></figure><h4>Issue 2: Pod Running but Readiness Failing</h4><blockquote><strong><em>Me</em></strong><em>: “Pod status is Running, but it’s never added to Service, why?”</em></blockquote><blockquote><strong><em>Claude</em></strong><em>: “Check Readiness probe:</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RDi6J8W2puu8O6eWYho05A.png" /></figure><blockquote><strong><em>Common Causes</em></strong><em>:</em></blockquote><ul><li><strong>Wrong probe path</strong> → Confirm app actually has /ready endpoint</li><li><strong>initialDelaySeconds too short</strong> → App hasn’t finished starting</li><li><strong>App actually has issues</strong> → Check container logs:</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*u4x2KOp0-Ho5QtjD8l4Jpw.png" /></figure><h4>Issue 3: Service Unreachable</h4><blockquote><strong><em>Me</em></strong><em>: “Service is created, but curl can’t connect?”</em></blockquote><blockquote><strong><em>Claude</em></strong><em>: “Troubleshoot step by step:</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vm4fg4xFLJPlCkr33y3AOA.png" /></figure><h3>Advanced Tip: Using AI to Analyze Logs</h3><h3>Real Case</h3><p>My Pod failed to start, logs showed:</p><pre>Error: failed to connect to database: dial tcp 10.0.1.5:3306: i/o timeout</pre><p>I directly sent the logs to Claude:</p><blockquote><strong><em>Me</em></strong><em>: “Claude, what’s causing this error?</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Bczk2mgX_LSjKx2O6CO7Nw.png" /></figure><blockquote><strong><em>Claude</em></strong><em>: “This is a </em><strong><em>network connectivity issue</em></strong><em>, possible causes:</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oLqLqHaqd1HU0f6tuQB5Mw.png" /></figure><p>Following this troubleshooting, I found the database address in ConfigMap was wrong (wrote external IP, should use Service name).</p><h3>Summary: Deployment Checklist</h3><p>I asked Claude to help me summarize a reusable checklist:</p><h3>📋 Pre-Deployment Checks</h3><ul><li>Image pushed to Alibaba Cloud Container Registry</li><li>ConfigMap configuration confirmed (environment variables, file paths)</li><li>Resource requests set reasonably (refer to historical monitoring data)</li><li>Health check paths correct (/healthz, /ready endpoints exist)</li><li>Namespace created</li><li>Dependent Services deployed (database, cache, etc.)</li></ul><h3>📋 Post-Deployment Verification</h3><ul><li>Deployment READY status is 3/3</li><li>Pod status is Running</li><li>Readiness probe passed (Pod added to Service)</li><li>Liveness probe passed (no restart loop)</li><li>Service Endpoints have Pod IPs</li><li>Service accessible from inside cluster</li><li>Application logs have no ERROR (kubectl logs)</li><li>Resource usage normal (kubectl top pod)</li></ul><h3>📋 Production-Grade Supplements</h3><ul><li>Set PodDisruptionBudget (prevent all Pods unavailable during update)</li><li>Configure HPA (Horizontal Pod Autoscaler)</li><li>Set resource Quota (prevent single app from exhausting cluster resources)</li><li>Configure monitoring and alerting (Prometheus/Alibaba Cloud ARMS)</li><li>Prepare rollback plan (kubectl rollout undo)</li></ul><h3>Next Steps: External Traffic Access</h3><p>Now the application is running inside the cluster, but external users still can’t access it. The next article will detail:</p><ul><li><strong>IngressClass three-tier architecture</strong></li><li>How to configure Alibaba Cloud ALB Ingress Controller</li><li>HTTPS/TLS certificate configuration</li><li>Multi-domain/path routing</li></ul><p><strong>Key Takeaways from This Article</strong>:</p><p>✅ Deployment order: ConfigMap → Deployment → Service</p><p>✅ Liveness detects alive, Readiness detects ready, both should be configured</p><p>✅ requests determines scheduling, limits determines throttling/eviction</p><p>✅ Service automatically tracks Pod changes through Endpoints</p><p>✅ Use Claude to analyze logs and troubleshoot</p><p>✅ Rolling update achieves zero downtime through maxUnavailable and maxSurge</p><h3>Appendix: Complete YAML (Ready to Use)</h3><pre>---<br>apiVersion: v1<br>kind: ConfigMap<br>metadata:<br>  name: demo-app-config<br>  namespace: production<br>data:<br>  APP_ENV: &quot;production&quot;<br>  LOG_LEVEL: &quot;info&quot;<br>  API_BASE_URL: &quot;https://api.example.com&quot;<br>​<br>---<br>apiVersion: apps/v1<br>kind: Deployment<br>metadata:<br>  name: demo-app<br>  namespace: production<br>spec:<br>  replicas: 3<br>  selector:<br>    matchLabels:<br>      app: demo-app<br>  template:<br>    metadata:<br>      labels:<br>        app: demo-app<br>    spec:<br>      containers:<br>      - name: app<br>        image: registry.cn-hangzhou.aliyuncs.com/demo/app:1.0.0<br>        ports:<br>        - containerPort: 8080<br>        envFrom:<br>        - configMapRef:<br>            name: demo-app-config<br>        resources:<br>          requests:<br>            cpu: 200m<br>            memory: 256Mi<br>          limits:<br>            cpu: 500m<br>            memory: 512Mi<br>        livenessProbe:<br>          httpGet:<br>            path: /healthz<br>            port: 8080<br>          initialDelaySeconds: 30<br>          periodSeconds: 10<br>        readinessProbe:<br>          httpGet:<br>            path: /ready<br>            port: 8080<br>          initialDelaySeconds: 10<br>          periodSeconds: 5<br>​<br>---<br>apiVersion: v1<br>kind: Service<br>metadata:<br>  name: demo-app<br>  namespace: production<br>spec:<br>  type: ClusterIP<br>  selector:<br>    app: demo-app<br>  ports:<br>  - port: 8080<br>    targetPort: 8080</pre><p><strong>Usage Instructions</strong>:</p><ol><li>Replace image address: registry.cn-hangzhou.aliyuncs.com/demo/app:1.0.0 → your image</li><li>Adjust resource limits: Modify CPU/memory based on actual needs</li><li>Modify health check paths: If your application uses different paths</li><li>Deploy: kubectl apply -f demo-app.yaml</li></ol><p><strong>Author</strong>: [TinyTwo] <strong>Creation Tool</strong>: This article was completed with the assistance of Claude AI, all flowcharts drawn using Mermaid <strong>Article Series</strong>: [2/5] — Next article “The Art of Traffic Management: IngressClass Three-Tier Architecture Explained”</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c5491310a038" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>