<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Elie hannouch on Medium]]></title>
        <description><![CDATA[Stories by Elie hannouch on Medium]]></description>
        <link>https://medium.com/@eliehannouch800?source=rss-79f152ec915------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*pEKPh9V-wTb16CBaKOYnsQ.jpeg</url>
            <title>Stories by Elie hannouch on Medium</title>
            <link>https://medium.com/@eliehannouch800?source=rss-79f152ec915------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 07:07:37 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@eliehannouch800/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[The agentic working set: designing memory, context and state in MongoDB]]></title>
            <link>https://medium.com/@eliehannouch800/the-agentic-working-set-designing-memory-context-and-state-in-mongodb-5dfd32dcdce0?source=rss-79f152ec915------2</link>
            <guid isPermaLink="false">https://medium.com/p/5dfd32dcdce0</guid>
            <category><![CDATA[mongodb]]></category>
            <category><![CDATA[agentic-ai-architecture]]></category>
            <category><![CDATA[enterprise-architecture]]></category>
            <category><![CDATA[rags]]></category>
            <category><![CDATA[enterprise-ai]]></category>
            <dc:creator><![CDATA[Elie hannouch]]></dc:creator>
            <pubDate>Thu, 24 Sep 2026 15:18:39 GMT</pubDate>
            <atom:updated>2026-09-24T15:18:39.871Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*O7wyEVmoX93EjRI7jC_crA.png" /></figure><blockquote>Agent memory is not one collection, and context is not a transcript. What an enterprise agent actually needs is a portfolio of states, deliberately separated, governed, and compiled at runtime.</blockquote><p>For most of the history of database engineering, a working set has had a precise and comfortable meaning: the documents and index pages accessed often enough that they should stay resident in memory if performance is to remain predictable. It was a property of the data and of the journeys people took through it. You could size it, watch it, and reason about it.</p><p>Agentic systems widen that boundary in a way the definition was never built for. An agent needs more than the records that answer a query. It needs a goal, a plan, the steps it has already taken, the conversation that produced the request, the policies that constrain it, the results of the tools it has called, the knowledge it retrieved, the authority it was delegated, a checkpoint it can resume from, and whatever it is supposed to remember afterwards.</p><p>The database is no longer holding only application state. It is holding most of the material from which the cognitive environment of an autonomous process gets assembled, one step at a time.</p><p>The industry’s default answer to this is a prototype pattern that has quietly become an architecture: store the chat history, embed it, retrieve the nearest chunks. It works impressively well in a demo and it breaks in almost every direction once the system meets an enterprise.</p><p>It breaks when sessions run for days rather than minutes. When agents work asynchronously and nobody is waiting. When several agents collaborate on one objective. When tools change business records and the agent’s recollection of those records goes stale mid-run. When a policy is superseded during execution. When permissions change between the moment something was remembered and the moment it is retrieved. When a decision has to be explained to an auditor, an appeal panel, or a regulator. When data carries a legal retention period. When tenants must not see each other’s context. When the embedding model is replaced. And, most dangerously, when an incorrect model output is written back as memory and becomes institutional fact.</p><p>The thesis of this article is a narrow one, and everything else follows from it:</p><blockquote><strong><em>The agentic working set is not everything the platform knows. It is the minimum authorised, relevant, current and trustworthy state required to complete the present objective.</em></strong></blockquote><p>Designing for that definition is mostly a data modelling problem, and MongoDB is a good foundation for it. Not because it can store everything in one flexible collection, but because it can represent many different kinds of state, each with its own shape, lifecycle and metadata, without forcing them into the same structure or the same guarantees.</p><p>It is worth naming the organisational failure mode early, because it is the one I see most often and it is not technical. Agent memory tends to arrive as a library default. A framework ships with a memory module, a team enables it, and within a quarter the organisation has a durable store of model-generated assertions about its customers that no domain owns, no retention policy covers, no review process touches and no auditor has ever seen. Nobody decided to build that. It accumulated.</p><p>Which is the honest framing of this whole subject. This is a data governance problem wearing an AI costume. The hard parts are ownership, provenance, authority, lifecycle and evidence, and they are hard parts the enterprise has solved before for other classes of data. The novelty is only that the writer is probabilistic, tireless and operating at machine speed.</p><h3>Two working sets, and the cost of confusing them</h3><p>It helps to separate the term into two layers that behave differently.</p><p>The <strong>physical working set</strong> is the familiar one: documents, indexes, vector index structures, cache pages, execution checkpoints, the operational records currently in play. Its enemy is read amplification and cache pressure.</p><p>The <strong>cognitive working set</strong> is what actually reaches the model on a given reasoning step: selected facts, instructions, memories, policies, tool contracts, retrieved passages. Its enemy is noise, staleness and unauthorised content.</p><p>Bad context architecture damages both at once, which is why the two are worth naming. Loading a whole session before every step wastes tokens and dilutes the model’s attention, and it forces the database to re-read documents it returned three steps ago. Large, ever-growing run documents hurt cache efficiency and network transfer. Repeated semantic searches compete with operational traffic for the same resources. Weak filtering lets stale or forbidden content into the prompt, which is a correctness and security failure rather than a performance one.</p><p>A rough shape for the cost of a single agent run:</p><pre>Data work per run<br>  ≈ steps<br>  × retrievals per step<br>  × documents examined per retrieval<br>  × average document size</pre><p>And across the platform, multiply by concurrent runs. Nothing in that expression is fixed by an engineer. Every term is chosen at runtime by the agent, which is what makes agentic capacity planning different from the API capacity planning most teams have done for a decade.</p><p>Note the second factor in particular. Documents examined is not documents returned. A retrieval that scans a thousand documents to return five costs the database a thousand documents of work, and the model sees none of that. The gap between examined and returned is one of the most useful numbers an agentic platform can track, and it comes straight out of query plan analysis.</p><p>There is a related and more cultural point. A large context window is a capability, not an instruction. The fact that a model accepts two hundred thousand tokens is not a reason to assemble two hundred thousand tokens. Every token assembled was retrieved from somewhere, and every retrieval was work the database did on behalf of a probabilistic process that may discard it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iFyvDGurb_yw9_JdUgAfQw.png" /><figcaption><em>Figure 1: the physical and cognitive working sets, and the cost model that connects them.</em></figcaption></figure><p>One more property belongs in the physical layer, because it catches teams late. Agents read their own writes constantly. A tool updates a case, the next step reads it back to verify, and a plan is formed from what came back. If those reads are routed to secondaries for scale, the agent can observe a state that precedes its own action and plan around it, which produces behaviour that looks like reasoning failure and is actually replication lag. Route an agent’s verification reads to the primary, or use a causally consistent session so the read observes the write, and reserve secondary reads for retrieval and analysis where staleness is tolerable. Read scaling is the correct instinct for agent traffic. It just has to stop at the boundary where the agent is checking its own effects.</p><h3>Memory is a label for several different data products</h3><p>The word “memory” is doing far too much work in most agent architectures. Pull it apart and the design becomes obvious.</p><p><strong>Authoritative business state</strong> is operational truth: the case, the application, the customer record. It is owned by a domain service, not by the agent. The model may read it and may propose changes to it, but it never owns it.</p><p><strong>Agent execution state</strong> is what allows a run to be paused, resumed, retried or transferred: objective, current plan, completed steps, checkpoints, budget consumed, lease, attempt count. It is short-lived, written frequently, and disposable once the run is closed and its evidence is preserved.</p><p><strong>Working memory</strong> is the scratchpad for the present objective: intermediate findings, partial computations, the shape of an answer being built. It is ephemeral by design. Promoting it to durable memory without a decision is how agents start believing their own drafts.</p><p><strong>Conversation state</strong> is the dialogue, which matters for reconstruction but rarely needs to be verbatim in full. Recent turns stay as they were said, older turns become summaries that retain references back to the original events.</p><p><strong>Episodic memory</strong> records that something happened: this customer was contacted on this date, this exception was granted, this approach failed. Retrieval is usually semantic plus temporal.</p><p><strong>Semantic memory</strong> holds reusable facts distilled from episodes. It is the most valuable and the most dangerous class, because a wrong entry here is repeated indefinitely.</p><p><strong>Procedural memory</strong> holds how to do things: a sequence that worked, a tool ordering, a checklist. In practice much of it should be promoted out of memory entirely and into code, prompts or tool definitions once it stabilises.</p><p><strong>Knowledge corpus</strong> is content the enterprise already owns: policies, manuals, regulations, product documentation. It is versioned, has effective dates, and belongs to content owners rather than to the agent.</p><p><strong>Tool-call history</strong> explains execution. Append-oriented, high volume, read mostly during debugging and audit.</p><p><strong>Plans and intermediate artifacts</strong> are the run’s own products: a draft assessment, a proposed set of changes, a ranked list.</p><p><strong>Policy and authorisation state</strong> is what the agent is allowed to do and see. It changes independently of the run and must be evaluated at retrieval time, not cached at session start.</p><p><strong>Audit and decision evidence</strong> is the record that lets the organisation explain an action afterwards. Immutable, retained by policy, and read rarely but at the worst possible moments.</p><p><strong>Telemetry and evaluation data</strong> measures whether any of this is working.</p><p>The useful discipline is to ask six questions of each class rather than to memorise the list:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Yiq4n6nO39utnOe5uNuRXA.png" /><figcaption><em>Figure 2: one word, several data products.</em></figcaption></figure><p>The last column is the one architects skip and then regret. Most state an agentic platform holds should never be in the active working set at all. It exists to explain the past, not to drive the present.</p><h3>Memory is not business truth, and the distinction has to be structural</h3><p>An agent may remember that a customer preferred to be contacted by email. The authoritative preference lives in the customer domain, and it may have changed this morning. An agent may remember that a case was marked urgent. The current status belongs to the case management system, and a human may have downgraded it an hour ago.</p><p>A memory is a contextual aid. It is not operational truth, and the moment a platform blurs the two it starts producing decisions that are confidently wrong in ways that are very hard to trace.</p><p>Three levels keep this clean:</p><p>→ <strong>Observation.</strong> What the agent saw, inferred or was told during a run. Cheap, plentiful, untrusted.</p><p>→ <strong>Candidate memory.</strong> An observation proposed for future reuse, carrying its provenance, confidence and intended scope.</p><p>→ <strong>Authoritative state.</strong> Validated data owned by a business domain, updated through that domain’s own commands and invariants.</p><p>Collapsing these produces a recognisable set of failures: stale decisions made on remembered values, conflicting records when the agent writes its recollection back, untraceable updates attributed to a service account, unauthorised persistence of data that should never have been retained, and feedback loops in which an agent retrieves its own earlier conclusion and treats it as corroboration.</p><p>The rule that prevents all of them is short. The model may propose a memory. A governed service decides whether it is admitted, updated, superseded, expired or rejected. That service is ordinary software with ordinary tests, and it is one of the highest-leverage components in the whole platform.</p><p>My position, stated plainly because it tends to be unpopular: most agentic platforms should ship with no durable memory at all, and earn it later. Execution state, conversation state and retrieval over governed knowledge will carry the first several use cases comfortably. Durable memory should arrive when there is a demonstrated case where remembering measurably improves an outcome, with an owner willing to answer for what is stored. Teams that start with memory enabled spend the following year trying to work out what is in it, and usually conclude that most of it was never used.</p><h3>Context is compiled, not accumulated</h3><p>If memory is a portfolio, something has to decide what to draw from it for each reasoning step. That component deserves a name and a place on the architecture diagram: the context compiler.</p><pre>Intent<br>  → Identity and delegated authority<br>  → Context policy for this objective<br>  → Deterministic retrieval (business state, policy, entitlements)<br>  → Semantic retrieval (knowledge, episodic and semantic memory)<br>  → Ranking, validation and de-duplication<br>  → Context assembly within budget<br>  → Agent execution step</pre><p>A compiled context typically contains the current objective, the last few conversation turns verbatim, a compact session summary, the current business state for the entities in play, the applicable policy version, approved long-term memories, retrieved knowledge with provenance, the contracts of the tools available at this step, the security constraints that apply, and the remaining execution budget.</p><p>Selection is a policy decision, not a similarity score. Relevance is only one criterion among relevance, authority, freshness, effective date, permission, tenant, jurisdiction, confidence, source quality, token budget and retrieval cost. A highly similar passage from a withdrawn policy should lose to a less similar passage from the policy currently in force, every time.</p><p>Two engineering details carry most of the benefit.</p><p>The first is projection. Context assembly should request the fields a goal needs rather than whole documents. It reduces bytes on the wire, bytes in cache, and tokens in the prompt at the same time.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2T33-yvMkIghkn1t3KM6uw.png" /><figcaption><em>Figure 3: context compiled per step, with retrieval bounded before it reaches the model.</em></figcaption></figure><p>The second is summarisation with references. Aggressive summarisation is the obvious way to keep long runs affordable, and it is also the obvious way to lose the one detail that mattered. The compromise that works is to summarise for the prompt while keeping identifiers for the underlying events in the manifest, so the system can recover the original evidence when a step needs it or an auditor asks. A summary that cannot be traced back to its sources is a lossy compression with no decompressor.</p><p>There is a third property of a context compiler that gets far less attention than it deserves: everything it assembles is data, and none of it is instruction. Retrieved knowledge, stored memories, tool output and conversation content can all contain text that reads like a command, whether placed there by an attacker or by an unlucky author. A context compiler that concatenates retrieved passages into the same undifferentiated block as the system instruction has built an injection surface and called it architecture. The assembly step should mark provenance and trust level explicitly, keep instructions structurally separate from content, and treat any instruction-like text inside retrieved material as what it is, which is a string in a document. This is also an argument for admitting memory through a governed pipeline rather than letting a run write directly, because durable memory is the most attractive injection target in the system: write once, retrieved forever, trusted by default.</p><p>Caching deserves a similar caution. Policy documents, tool contracts and entitlement sets are stable enough to cache, and caching them removes a great deal of repeated read work. Business state and permissions are not. A cached entitlement that survives a role change is a disclosure waiting for a schedule, and a cached business record that survives a tool write is the replication lag problem with a longer memory. Cache what is versioned and slow moving, key the cache by version, and invalidate on publication rather than on a timer.</p><h3>Context as a versioned runtime artifact</h3><p>If context is compiled, then each compilation is an artifact, and artifacts can be recorded. This is the single design decision that most improves an agentic platform’s auditability.</p><pre>{<br>  &quot;_id&quot;: &quot;context-9281&quot;,<br>  &quot;tenantId&quot;: &quot;tenant-41&quot;,<br>  &quot;runId&quot;: &quot;run-882&quot;,<br>  &quot;step&quot;: 7,<br>  &quot;objective&quot;: &quot;Assess application eligibility&quot;,<br>  &quot;identitySnapshot&quot;: {<br>    &quot;subjectId&quot;: &quot;user-72&quot;,<br>    &quot;agentId&quot;: &quot;eligibility-agent&quot;,<br>    &quot;agentVersion&quot;: &quot;4.2&quot;,<br>    &quot;delegatedScopes&quot;: [&quot;application:read&quot;, &quot;assessment:propose&quot;]<br>  },<br>  &quot;sources&quot;: [<br>    { &quot;type&quot;: &quot;business-record&quot;, &quot;sourceId&quot;: &quot;application-718&quot;, &quot;version&quot;: 12, &quot;retrievedAt&quot;: &quot;...&quot; },<br>    { &quot;type&quot;: &quot;policy&quot;, &quot;sourceId&quot;: &quot;policy-2026-04&quot;, &quot;version&quot;: 4, &quot;effectiveFrom&quot;: &quot;...&quot; },<br>    { &quot;type&quot;: &quot;knowledge-chunk&quot;, &quot;sourceId&quot;: &quot;chunk-55192&quot;, &quot;docVersion&quot;: 7, &quot;score&quot;: 0.81 }<br>  ],<br>  &quot;memoryIds&quot;: [&quot;memory-81&quot;, &quot;memory-97&quot;],<br>  &quot;knowledgeVersion&quot;: &quot;kb-2026-09-12&quot;,<br>  &quot;embeddingVersion&quot;: &quot;embed-v5&quot;,<br>  &quot;promptVersion&quot;: &quot;assessment-v9&quot;,<br>  &quot;model&quot;: &quot;model-2026-06&quot;,<br>  &quot;tokenBudget&quot;: 18000,<br>  &quot;tokensUsed&quot;: 11240,<br>  &quot;createdAt&quot;: &quot;...&quot;<br>}</pre><p>With this in place the platform can answer a question that is otherwise unanswerable six months later: what was this agent allowed to see, and what did it actually see, at the moment it made that call.</p><p>That supports audit, incident analysis, appeals, regulatory evidence, model comparison, prompt regression testing and rollback. One honest caveat belongs here, because the claim is often overstated. Recording the manifest lets you reconstruct the input conditions of a decision. It does not guarantee that replaying those conditions reproduces the same output, because the model is probabilistic. Reconstructing inputs and reproducing outputs are different properties, and only the first is within the database’s gift.</p><h3>Collections follow lifecycles, not entities</h3><p>The temptation with a document database is to put the run and everything about it in one document. It models beautifully and it fails in production for reasons that are entirely predictable.</p><p>Steps and tool calls are unbounded arrays. Documents that grow are documents that get rewritten, and rewrites of a large document are expensive. Multikey indexes over big arrays add their own pressure. The 16 MB document limit eventually arrives, usually during an incident rather than a test. Retention differs: execution detail should expire while decision evidence must not. Access control differs: a support engineer may need tool traces without any right to see business content. Update rates differ: a checkpoint changes constantly while the run’s identity never does. And a single hot document under concurrent writers is contention by design.</p><p>The principle that survives contact with production:</p><blockquote><strong><em>Embed state that is bounded, retrieved together, and shares a lifecycle. Reference state that grows, changes, secures, expires or scales independently.</em></strong></blockquote><p>A workable topology, with the caveat that exact boundaries depend on scale, ownership and regulation:</p><p>→ agentRuns, agentSteps, contextManifests</p><p>→ workingMemory, memoryCandidates, durableMemories</p><p>→ knowledgeDocuments, knowledgeChunks</p><p>→ toolExecutions, decisionEvidence, businessEvents, evaluationResults</p><p>An agent run keeps only what is bounded and current:</p><pre>{<br>  _id: &quot;run-882&quot;,<br>  tenantId: ObjectId(&quot;...&quot;),<br>  agentId: &quot;eligibility-agent&quot;,<br>  agentVersion: &quot;4.2&quot;,<br>  schemaVersion: 3,<br>  objective: &quot;Assess application eligibility&quot;,<br>  status: &quot;RUNNING&quot;,<br>  version: 17,<br>  currentStep: 7,<br>  plan: { revision: 2, pendingTools: [&quot;policy-lookup&quot;] },<br>  lease: { owner: &quot;worker-9&quot;, expiresAt: ISODate(&quot;...&quot;) },<br>  budget: { maxSteps: 25, maxToolCalls: 40, maxDbOps: 300, maxVectorSearches: 20 },<br>  usage:  { steps: 7, toolCalls: 11, dbOps: 96, vectorSearches: 4 },<br>  correlationId: &quot;journey-551&quot;,<br>  startedAt: ISODate(&quot;...&quot;),<br>  expiresAt: ISODate(&quot;...&quot;)<br>}</pre><p>Tool executions are appended, never accumulated in the run:</p><pre>{<br>  _id: &quot;tool-881&quot;,<br>  runId: &quot;run-882&quot;,<br>  step: 7,<br>  tenantId: ObjectId(&quot;...&quot;),<br>  tool: { name: &quot;policy-lookup&quot;, version: &quot;2.1&quot; },<br>  request: { policyId: &quot;policy-2026-04&quot; },<br>  outcome: &quot;SUCCESS&quot;,<br>  durationMs: 214,<br>  dbOps: 3,<br>  occurredAt: ISODate(&quot;...&quot;)<br>}</pre><p>And a durable memory carries everything needed to trust it later, or to stop trusting it:</p><pre>{<br>  _id: &quot;memory-97&quot;,<br>  tenantId: ObjectId(&quot;...&quot;),<br>  subjectId: &quot;customer-3391&quot;,<br>  scope: &quot;TENANT_SHARED&quot;,<br>  type: &quot;SEMANTIC&quot;,<br>  statement: &quot;Prefers written summaries before scheduled calls&quot;,<br>  confidence: 0.82,<br>  provenance: {<br>    runId: &quot;run-774&quot;,<br>    step: 4,<br>    evidence: [&quot;tool-6612&quot;, &quot;conversation-turn-88&quot;],<br>    admittedBy: &quot;memory-service&quot;,<br>    admittedAt: ISODate(&quot;...&quot;)<br>  },<br>  validity: { from: ISODate(&quot;...&quot;), until: null },<br>  supersedes: &quot;memory-61&quot;,<br>  supersededBy: null,<br>  classification: &quot;INTERNAL&quot;,<br>  embedding: [/* ... */],<br>  embeddingVersion: &quot;embed-v5&quot;,<br>  reviewedAt: null<br>}</pre><p>The schemaVersion field on long-lived documents is not decoration. Agent platforms change shape every few weeks, and the document model’s tolerance for mixed shapes is only an advantage if the application knows which shape it is reading.</p><h3>Execution state has to survive a crashed worker</h3><p>The difference between a demo and a platform is what happens when a process dies halfway through a run.</p><p>A resumable run needs the objective, the current plan, completed steps, pending tool calls, checkpoints, an execution lease, an attempt count, budget consumed, cancellation state, any pending human approval, and correlation identifiers. The design question is where to checkpoint. Saving every token is expensive and pointless. Checkpointing at step boundaries, before and after any tool call that has external effects, is enough to resume without repeating work that mattered.</p><p>Two workers acquiring the same run is the classic failure. Optimistic concurrency with a version field and a lease solves it without distributed locks:</p><pre>const claimed = await agentRuns.updateOne(<br>  {<br>    _id: runId,<br>    version: expectedVersion,<br>    $or: [<br>      { &quot;lease.owner&quot;: workerId },<br>      { &quot;lease.expiresAt&quot;: { $lt: new Date() } }<br>    ]<br>  },<br>  {<br>    $set: {<br>      &quot;lease.owner&quot;: workerId,<br>      &quot;lease.expiresAt&quot;: new Date(Date.now() + 30000),<br>      currentStep: nextStep<br>    },<br>    $inc: { version: 1 }<br>  }<br>);</pre><pre>if (claimed.matchedCount === 0) {<br>  // another worker advanced this run, or the lease is still held: stand down<br>  return;<br>}</pre><p>The document update is atomic, so exactly one worker wins. The loser does nothing, which is the correct behaviour and a great deal safer than two agents pursuing the same objective with different plans.</p><p>Tool calls need the complementary control. Retries are normal in agentic systems, so every externally visible command carries an idempotency key on a unique index:</p><pre>db.commands.createIndex({ idempotencyKey: 1 }, { unique: true });</pre><p>A duplicate insert fails on the index rather than repeating the business action. Retryable writes protect a single operation against a transient network blip, and multi-document transactions protect invariants that span documents, but neither of them makes a business action idempotent across a fresh attempt by a new worker. That is application design, and it is the difference between an agent that recovers and an agent that sends the payment twice.</p><h3>Memory admission is a governed write path</h3><p>“The agent learned something” has to translate into a data mutation somebody can defend.</p><pre>Observation<br>  → Candidate memory<br>  → Validation against schema and permitted categories<br>  → Sensitive-data and classification check<br>  → De-duplication against existing memories<br>  → Contradiction detection against business state and prior memories<br>  → Authority check: may this agent remember this, for this tenant, for this purpose<br>  → Human or policy approval where required<br>  → Durable memory, with provenance and validity</pre><p>The controls worth enforcing before anything becomes long-lived: an explicit business purpose, a permitted memory category, source provenance, a confidence value, consent where the jurisdiction requires it, tenant ownership, a data classification, a retention and expiry rule, contradiction handling, supersession rather than silent overwrite, revocation, and propagation of deletion requests.</p><p>Consolidation deserves a note. Several observations often deserve to become one curated fact, and that is a genuine improvement in quality and cost. It becomes a liability the moment consolidation drops the links to the underlying evidence, because the platform then holds a confident assertion with no way to verify it. Consolidate the statement, keep the references.</p><p>Supersession rather than overwrite matters for the same reason. When a newer memory replaces an older one, both remain, linked, with validity periods. Retrieval reads the current one. Audit can still see what was believed last quarter, which is exactly the question that gets asked when a decision is challenged.</p><p>Two honest notes about this pipeline, because it is easy to draw and hard to run.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_6fuuggzt3pS2xACb6OwpQ.png" /><figcaption><em>Figure 4: the memory admission pipeline, from observation to durable memory.</em></figcaption></figure><p>Contradiction detection is genuinely difficult. Comparing a proposed memory against business state is tractable when the memory maps to a field with a known meaning. It is much weaker for free-form assertions, where a new memory may contradict an old one in ways no deterministic check will catch. The pragmatic approach is to make the memory schema narrow enough that most entries map to typed, checkable statements, and to treat free-text memory as a low-confidence category that never influences a consequential decision on its own.</p><p>Human review does not scale, and pretending otherwise produces a review queue nobody reads. Reserve approval for memory classes with real consequence, such as anything about a person, anything that will affect eligibility or pricing, and anything classified above internal. Everything else should be admitted automatically under policy, with sampling for quality rather than review for volume. A control that is skipped because it is too slow is worse than a control that was scoped honestly.</p><h3>Retrieval combines semantic relevance with deterministic boundaries</h3><p>An enterprise retrieval is never “find things like this prompt”. It is closer to: find semantically relevant material, within this tenant, visible to this identity, valid in this jurisdiction, in force at this time, from approved sources, under the current embedding version.</p><p>In MongoDB those constraints belong inside the vector search as pre-filters, and every field used has to be declared as a filter field in the index definition:</p><pre>{<br>  &quot;fields&quot;: [<br>    { &quot;type&quot;: &quot;vector&quot;, &quot;path&quot;: &quot;embedding&quot;, &quot;numDimensions&quot;: 1536, &quot;similarity&quot;: &quot;cosine&quot; },<br>    { &quot;type&quot;: &quot;filter&quot;, &quot;path&quot;: &quot;tenantId&quot; },<br>    { &quot;type&quot;: &quot;filter&quot;, &quot;path&quot;: &quot;accessGroups&quot; },<br>    { &quot;type&quot;: &quot;filter&quot;, &quot;path&quot;: &quot;status&quot; },<br>    { &quot;type&quot;: &quot;filter&quot;, &quot;path&quot;: &quot;effectiveFrom&quot; },<br>    { &quot;type&quot;: &quot;filter&quot;, &quot;path&quot;: &quot;jurisdiction&quot; },<br>    { &quot;type&quot;: &quot;filter&quot;, &quot;path&quot;: &quot;embeddingVersion&quot; }<br>  ]<br>}</pre><pre>db.knowledgeChunks.aggregate([<br>  {<br>    $vectorSearch: {<br>      index: &quot;knowledge_vector_idx&quot;,<br>      path: &quot;embedding&quot;,<br>      queryVector: queryEmbedding,<br>      numCandidates: 200,<br>      limit: 10,<br>      filter: {<br>        $and: [<br>          { tenantId: ctx.tenantId },<br>          { accessGroups: { $in: ctx.identity.groups } },<br>          { status: &quot;PUBLISHED&quot; },<br>          { effectiveFrom: { $lte: new Date() } },<br>          { jurisdiction: ctx.jurisdiction },<br>          { embeddingVersion: &quot;embed-v5&quot; }<br>        ]<br>      }<br>    }<br>  },<br>  { $project: { text: 1, sourceId: 1, docVersion: 1, score: { $meta: &quot;vectorSearchScore&quot; } } }<br>], { comment: ctx.opContext });</pre><p>The distinction between pre-filtering and post-filtering is not stylistic. MongoDB applies pre-filters as it traverses the index rather than afterwards, so candidate slots are spent only on documents the caller may actually see. Filtering after retrieval with a $match can return fewer results than requested when most candidates are excluded, which silently degrades answer quality, and it means forbidden documents were read before being discarded. Relevance never overrides authorisation, and the cheapest way to guarantee that is to make it structurally impossible for the search to consider material the identity cannot access.</p><p>Two practical notes. The filter option supports a limited set of operators against BSON types, so security predicates need to be expressible as simple equality, range and set membership. Design the metadata for that, rather than hoping to run arbitrary logic at retrieval time. And pre-filters interact with numCandidates: a highly selective filter usually needs a higher candidate count to return a full result set, so tune it against a measured recall benchmark rather than a rule of thumb.</p><p>There is also a modelling boundary that gets blurred too often. A business document and its retrieval chunks are not the same object. Chunks are a derived representation with their own lifecycle, their own version, and their own relationship back to a specific version of the source. Keeping operational records and chunked knowledge in separate collections, joined by explicit version references, is what makes it possible to answer whether a retrieved passage still reflects the document it came from.</p><h3>Embeddings and knowledge evolve, and the metadata has to know</h3><p>Semantic retrieval creates a category of data that is derived, opaque and quietly perishable. A vector does not know that its source was withdrawn last week. It will keep matching.</p><p>Each chunk should therefore carry the facts that let deterministic logic override similarity: source identifier and version, chunk version, embedding model identifier, embedding timestamp, a content hash, the effective period, a supersession reference, and the access-policy version in force when it was indexed.</p><p>With those fields present, the awkward operational events become manageable rather than frightening.</p><p>When the embedding model changes, write the new vectors alongside the old under a new embeddingVersion, backfill in batches, evaluate recall on a held-out set, then switch the retrieval filter. A staged migration costs storage for a period. A blind replacement costs correctness for a period, and nobody can tell you how long that period is.</p><p>When chunking strategy changes, treat it as a new chunk version of the same source rather than an edit, for the same reason.</p><p>When source content changes or a document is withdrawn, the chunks have to be invalidated by the same pipeline that publishes them. A withdrawn policy with live chunks is the most common way an agentic platform quotes a rule that no longer exists.</p><p>When a tenant’s permissions change, the access metadata on the chunks has to be updated, and any cached retrieval results invalidated. Caching retrieval output across permission changes is a leak waiting for a schedule.</p><h3>Placement, keys and query targeting</h3><p>Agent-generated collections have an unhelpful natural key. Run identifiers and timestamps both increase monotonically, and a monotonic shard key sends every insert to the same chunk on the same shard, which turns a burst of agent activity into a single-shard hotspot while the rest of the cluster idles.</p><p>For tenanted platforms the pragmatic key leads with tenant and adds a hashed component, which distributes writes while keeping tenant-scoped reads targeted to a small number of shards:</p><pre>sh.shardCollection(&quot;agents.durableMemories&quot;, { tenantId: 1, subjectId: &quot;hashed&quot; });</pre><p>Targeting is the property to protect. A query that carries the shard key prefix reaches one shard. A query that does not becomes a scatter-gather across all of them, and agent workloads issue enough queries that the difference shows up as a platform characteristic rather than a slow endpoint. Retrieval paths in particular should be designed so that tenant is always present, which is the same requirement security already imposed.</p><p>Two caveats worth stating. Leading with tenant helps distribution across many tenants and does nothing for a single tenant large enough to be a hotspot on its own, which eventually needs either a finer key or dedicated placement. And vector search has its own topology: on Atlas the search workload can be moved to dedicated nodes, which is a different lever from sharding and solves a different problem. Distributing data and isolating a workload are not substitutes for each other.</p><h3>Identity and delegated authority have to reach the data layer</h3><p>Once agents act, “who changed this” becomes a chain rather than a name:</p><pre>Human or system initiator<br>  → Supervising agent<br>  → Specialist agent<br>  → Tool<br>  → MongoDB operation</pre><p>Every material state change should retain the initiating principal, the acting agent and its version, the delegated scope, the tool name and version, the run and step identifiers, any approval reference, the business purpose, the affected entity, a before-and-after reference, and a timestamp. A write attributed to agent-service retains none of that and is not an audit record.</p><p>Three MongoDB-specific points make this work rather than remain an aspiration.</p><p>Write the actor into the document, not only into a log. Change streams report what changed, and with pre- and post-images enabled on the collection they can report the full document before and after the change, which is excellent for building evidence pipelines. What they cannot do is invent an actor that was never stored. If the acting agent and its authority are in the document, a change stream consumer can construct a complete audit trail. If they are not, nothing downstream can recover them.</p><p>Attach execution context to every operation. A comment carrying run, agent and tool-call identifiers surfaces in the profiler, the slow query log and $currentOp, which is how an operator identifies and stops a misbehaving agent during an incident rather than guessing from query shapes.</p><p>Give each agent class its own database role. A read-only research agent and a case-resolution agent with delegated write authority should not share credentials, because the database is the last boundary that still holds when the orchestration layer misbehaves.</p><h3>Budget the data work, and isolate the workload classes</h3><p>One human request can trigger several planning steps, repeated retrieval, multiple semantic searches, tool reads and writes, delegated agents, memory persistence, evaluation traces and audit records. The total is not visible from any single component.</p><pre>Total data work<br>  = runs × steps × retrievals per step × documents per retrieval<br>  + tool operations<br>  + state writes<br>  + evidence writes</pre><p>Every term needs a ceiling, and the ceilings have to live outside the model. Step budgets, retrieval budgets, maximum context size, maximum result counts, execution deadlines, delegation depth, retry budgets, maxTimeMS on every agent-issued query, bounded connection pools, work queues, per-tenant and per-agent rate limits, caches for stable policy and tool metadata, cancellation propagation, and circuit breakers on tools that are failing. A prompt instruction to stop after ten steps is a suggestion. A budget enforced at the tool gateway is a control, because the model has no other route to the data.</p><p>The workload classes then need separating, because they have genuinely different requirements:</p><p>→ Interactive user journeys and operational domain transactions, which are latency sensitive and business critical</p><p>→ Agent execution state, which is small, hot and tolerant of weaker durability</p><p>→ Semantic retrieval, which is bursty and resource hungry</p><p>→ Knowledge ingestion, re-embedding and memory consolidation, which are batch workloads with no user waiting</p><p>→ Evaluation, analytics, audit and retention jobs, which are heavy and entirely deferrable</p><p>Separate connection pools per class stop agent concurrency from exhausting the connections transactional services depend on. Read preference tags route analytical and evaluation reads to dedicated analytics nodes. On Atlas, dedicated search nodes separate vector and full-text workloads from the operational database. Write concern can vary by class: w: &quot;majority&quot; for business state, a lighter concern for telemetry whose loss is an inconvenience rather than an incident.</p><p>Shared infrastructure is usually the right starting point. The signals that justify separate clusters or dedicated placement are concrete: a batch re-embedding job that cannot be scheduled outside business hours, a tenant with residency requirements, a single tenant large enough to become a hotspot on its own, or an evaluation workload whose read volume rivals production traffic.</p><h3>Tenancy is a memory problem before it is an infrastructure problem</h3><p>Conventional multi-tenancy is well understood: put tenantId on the record, lead the compound index with it, filter every query. Memory introduces a subtler boundary, because semantic retrieval can reach content that no query predicate ever named.</p><p>Every memory and every chunk carries its tenant. Vector indexes declare tenant as a filter field, and no retrieval path exists that does not apply it. Beyond that, memory needs a scope distinction that ordinary records do not: user-private, tenant-shared, and cross-tenant reference material are three different things, and only the last should ever be visible outside its origin.</p><p>The rest follows from treating memory as regulated data: per-tenant retention and encryption policy, tenant export and deletion that reaches derived representations, isolation for a noisy agent so one tenant’s automation cannot consume another tenant’s capacity, placement for large tenants, and regional residency where the law requires it. Field-level encryption or queryable encryption is worth considering for the most sensitive memory content, with the honest trade-off that encrypted fields constrain what you can query and rank on.</p><p>An agent serving many tenants may share capability, code and prompts. It must not share ungoverned memory.</p><h3>Retention and deletion have to reach derived data</h3><p>State moves through a lifecycle: created, active, summarised or consolidated, superseded, archived, expired or deleted. Each transition is a design decision rather than an accident of storage.</p><p>TTL indexes are the right mechanism for genuinely disposable execution state, and their behaviour needs to be understood precisely. A background thread removes expired documents, so deletion is not instantaneous and should never be treated as an exact business deadline. Under load or during failover the lag grows. If a legal obligation requires deletion by a specific time, TTL is a cleanup mechanism supporting that obligation, not the control that satisfies it.</p><p>The deeper problem is derived data. Deleting a source document without invalidating its chunks, embeddings, summaries and the memories distilled from it leaves a shadow copy of the information, retrievable by exactly the semantic path the deletion was meant to close. Right-to-delete propagation has to traverse the derivation graph: source, chunks, vectors, summaries, memories, evaluation samples and caches.</p><p>Restoration has the same shape in reverse. A restored backup can reintroduce memories that were revoked and embeddings that were superseded. Restore procedures for agentic platforms need a reconciliation step, not just a point-in-time recovery. Derived data is still governed data.</p><h3>Observability has to explain decisions, not just load</h3><p>The chain worth instrumenting runs from business outcome all the way to query shape:</p><pre>Business outcome<br>  → Agent run<br>  → Step<br>  → Context manifest<br>  → Retrieval query<br>  → Tool call<br>  → MongoDB query shape<br>  → State mutation</pre><p>Useful metrics fall into two families. Cost: steps per outcome, database operations per outcome, retrievals per step, documents examined versus returned, context size, repeated retrieval rate within a run, vector search latency, checkpoint write rate, run duration, stalled runs, and cost per successful outcome. Quality: memory hit utility, stale memory retrieval rate, superseded memory usage, unauthorised retrieval attempts, tool retry rate, and human escalation rate.</p><p>The distinction that matters: monitoring tells you the database is busy. Agentic operational intelligence tells you which agent version, context policy, tenant, prompt version or retrieval pattern made it busy. The first gets you through the night. The second stops the next one happening.</p><p>Memory quality deserves evaluation in its own right, separate from retrieval accuracy. Was the retrieved memory relevant, current, authorised and authoritative? Did it improve the outcome or merely fill the prompt? Did it contradict business state? Should it have been forgotten? Did the model over-trust a low-confidence memory? Did a memory create a feedback loop by confirming a conclusion the agent itself wrote?</p><p>Recall is the wrong objective function. The goal is the smallest trustworthy context sufficient for a correct outcome, and the metrics should reward compression that preserves outcomes rather than retrieval that fills windows.</p><h3>Failures the architecture has to survive</h3><p>None of these is exotic. Every one of them follows directly from something earlier in this article, which is the point: they are not a checklist bolted on at the end, they are what the separation of state classes was for.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Hc-yUk5JA_8ieQXLFbN6SQ.png" /><figcaption><em>Figure 5: failure scenarios and the control each one requires.</em></figcaption></figure><h3>The target architecture</h3><pre>Channels and business events<br>        ↓<br>Agent runtime and orchestrator<br>        ↓<br>Identity, delegation and policy gate<br>        ↓<br>Context compiler<br>   ├── Session and working state<br>   ├── Governed memory<br>   ├── Knowledge retrieval<br>   ├── Authoritative business APIs<br>   └── Tool registry<br>        ↓<br>Model and planning layer<br>        ↓<br>Proposed action<br>        ↓<br>Deterministic validation and approval<br>        ↓<br>Domain command and MongoDB state transition<br>        ↓<br>Evidence, evaluation and memory admission</pre><p>Read it as a set of ownership boundaries rather than a pipeline. The policy gate owns what the agent may see and do. The context compiler owns what reaches the model. The domain owns business truth. The memory service owns what becomes durable. The evidence layer owns what can be explained afterwards.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WKACDbG1p483S2DcNsrEKw.png" /><figcaption><em>Figure 6: the target architecture, read as ownership boundaries.</em></figcaption></figure><p>The model owns none of it. That is deliberate, and it is the property that makes the platform durable: models will be replaced, probably several times, and every replacement should be a configuration change rather than an architectural one. If swapping the model would lose your memory policy, your authority model or your evidence, then the model was holding state it should never have held.</p><h3>Who owns what</h3><p>Architecture diagrams describe components. Platforms fail on ownership, so it is worth being explicit about which team answers for each boundary.</p><p>→ <strong>Domain teams</strong> own business state and the commands that change it. They also own the contracts agents call, which means they get a say in what an agent is permitted to propose.</p><p>→ <strong>The platform team</strong> owns the agent runtime, execution state, budgets and the tool gateway. This is where enforcement lives, so it needs the authority to refuse an agent that will not declare its costs.</p><p>→ <strong>A memory owner</strong> owns the admission policy, the memory schema, retention and revocation. This role is almost always missing at the start, and its absence is what turns memory into an unowned liability.</p><p>→ <strong>Content owners</strong> own the knowledge corpus and its lifecycle, including withdrawal. If publication is governed and withdrawal is not, retrieval will quote a rule that no longer exists.</p><p>→ <strong>Security and compliance</strong> own classification, tenancy boundaries, evidence retention and the definition of what must be explainable.</p><p>The pattern to avoid is the one where the AI team owns all of it. Not because they are not capable, but because memory, knowledge and business truth outlive any particular agent, and state owned by a project has a habit of expiring with the project.</p><h3>Where this is too much architecture</h3><p>Not every agentic system needs this, and an architect who cannot say when it is excessive is selling rather than advising.</p><p>A single collection holding conversation and a few embeddings is entirely reasonable for an internal assistant with short sessions, one tenant, no autonomous writes, no regulated content and a small blast radius if it is wrong. The cost of the architecture above is real: more collections, more services, more policy, more latency in the assembly step, and a memory pipeline somebody has to run.</p><p>The signals that a system has outgrown the simple pattern are specific, and they arrive in a fairly consistent order.</p><p>→ The agent gains write authority over business records. This one changes everything, because a wrong memory can now become a wrong action.</p><p>→ A second tenant appears, or a second agent shares the same store.</p><p>→ Sessions become long-running or asynchronous, so context must be reconstructed rather than held.</p><p>→ Someone outside the team asks why a decision was made, and the answer is a shrug.</p><p>→ Content has an effective date, a withdrawal process or a jurisdiction.</p><p>→ Retrieval cost or latency becomes a line item rather than a rounding error.</p><p>The sensible adoption order mirrors the dependency between these controls rather than their difficulty. Budgets and identity first, because they bound damage and make everything explainable. Then the separation of execution state from evidence, so runs recover and actions can be defended. Then filtered retrieval, because that is where disclosure risk lives. Then governed memory admission, which is the most expensive to build and the easiest to defer. Then context manifests and evaluation, which are what make the platform improvable rather than merely working.</p><p>Teams that reverse this order tend to build the memory pipeline first, because it is the most interesting part, and then discover they cannot explain what any of it did.</p><h3>Closing position</h3><p>Everything above resolves into a few commitments that are worth stating plainly.</p><p>Context is a runtime product, not a transcript. Memory is a governed lifecycle, not an embedding. Business truth stays with the domain that owns it. Execution state exists to make runs recoverable and actions idempotent. Retrieval combines semantic relevance with deterministic authorisation, and the second is not negotiable when the two disagree. Every consequential decision leaves reconstructable evidence. Every agent has a data work budget. And the model remains replaceable.</p><p>MongoDB’s contribution to this is specific, and worth stating without inflation. It can represent evolving state, rich metadata and embeddings in the same documents, which keeps semantic discovery connected to enterprise context and removes a whole class of synchronisation problems. It offers the primitives these patterns need: atomic document updates for concurrency control, unique indexes for idempotency, TTL indexes for disposable state, pre-filtered vector search for bounded retrieval, change streams for evidence, transactions where invariants span documents. What it does not offer is a reason to collapse all of that state into one collection because the schema would allow it.</p><p>None of this is a reason to build all of it on day one. It is a reason to know which of it you are deliberately deferring, and to be able to say what would make you stop deferring.</p><p>The mature agentic platform will not be distinguished by how much it remembers. It will be distinguished by how deliberately it decides what may be remembered, what must be retrieved, what remains authoritative, and what an agent is allowed to do with any of it.</p><p>The agentic working set should contain neither everything the enterprise knows nor everything the agent has ever seen. It should contain the smallest trustworthy state required to act correctly, and enough evidence to explain afterwards why that action occurred.</p><h3>References</h3><p>→ MongoDB Vector Search, $vectorSearch stage and pre-filtering: <a href="https://www.mongodb.com/docs/atlas/atlas-vector-search/vector-search-stage/">https://www.mongodb.com/docs/atlas/atlas-vector-search/vector-search-stage/</a></p><p>→ TTL indexes and the behaviour of the background deletion process: <a href="https://www.mongodb.com/docs/manual/core/index-ttl/">https://www.mongodb.com/docs/manual/core/index-ttl/</a></p><p>→ Change streams, including document pre- and post-images, transactions, retryable writes, read and write concerns, schema versioning, query plans and profiling, sharding and query targeting, connection pooling, encryption, and backup and restore: MongoDB Manual and Atlas documentation at <a href="https://www.mongodb.com/docs">https://www.mongodb.com/docs</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5dfd32dcdce0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The next million users won’t all be human: scaling MongoDB for agentic workloads]]></title>
            <link>https://medium.com/@eliehannouch800/the-next-million-users-wont-all-be-human-scaling-mongodb-for-agentic-workloads-ead9ffd2fe22?source=rss-79f152ec915------2</link>
            <guid isPermaLink="false">https://medium.com/p/ead9ffd2fe22</guid>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[software-architecture]]></category>
            <category><![CDATA[mongodb]]></category>
            <category><![CDATA[data-engineering]]></category>
            <category><![CDATA[enterprise-ai]]></category>
            <dc:creator><![CDATA[Elie hannouch]]></dc:creator>
            <pubDate>Tue, 22 Sep 2026 17:42:40 GMT</pubDate>
            <atom:updated>2026-09-23T12:23:20.922Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8gs66NFowJycO2zZH7FEuw.png" /></figure><blockquote>Agents turn one request into a workload graph that grows while it runs. Here is how to design MongoDB so that graph stays bounded, attributable and reversible.</blockquote><h3>02:14</h3><p>At 02:14 on a Tuesday, the on-call engineer for a government services platform is paged.</p><p>CPU on the primary of the operational cluster is at 94%. Read latency on the cases collection has climbed from 4 milliseconds to 900. Citizen-facing transactions are starting to time out.</p><p>Nobody is using the platform. It is the middle of the night, and human traffic is at its daily low.</p><p>The load is coming from a case-resolution agent. Forty minutes earlier, a document validation service it depends on began returning intermittent errors. The agent did exactly what it was designed to do. It retried. Then it re-read the case to check whether anything had changed. Then it searched the knowledge base for a policy that might explain the error. Then it retried again.</p><p>Across 3,000 scheduled overnight runs, that loop generated more database work in forty minutes than the platform normally sees in its busiest hour.</p><p>The scenario is illustrative. Every element of it, though, is a pattern that agentic platforms have to be designed against, and none of it would have been caught by a conventional load test. The users did not change. The cluster did not change. The only thing that changed was that some of the platform’s users were no longer people.</p><p>This article is about designing MongoDB for that reality.</p><h3>The persona that changed the maths</h3><p>For most of the history of application architecture, database demand was anchored to human behaviour.</p><p>A person opened a page, submitted a form, searched for a record or approved a request. Even when one action triggered several database operations, the human was the origin of demand, and an engineer had designed the path that demand followed. You could draw it on a whiteboard. You could load-test it. You could predict it.</p><p>Agents break that assumption.</p><p>An agent is not another interface through which a user submits a query. It is a workload generator. At runtime, it decides what to retrieve, how many steps to take, which tools to call, when to retry, what to remember and whether to hand part of the job to another agent.</p><p>A single human request now looks less like a line and more like a tree:</p><pre>Human intent<br>  └── Planning agent<br>        ├── Retrieve user and tenant context<br>        ├── Semantic search over knowledge<br>        ├── Read operational state<br>        ├── Call tool<br>        │     └── Further reads and writes<br>        ├── Evaluate result<br>        ├── Revise plan<br>        ├── Delegate to validation agent<br>        │     └── Its own reads, searches and tools<br>        └── Persist memory</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IBMx7frNVZWHOhMWkEfHXQ.png" /><figcaption><em>Figure 1: the same business intent, served by an API and by an agent.</em></figcaption></figure><p>A conventional request might perform three or four MongoDB operations. An agentic request might perform fifty. And two identical requests can produce very different costs. One run finishes in three steps. Another loops through twenty because it cannot reach enough confidence to stop.</p><p>That gives scalability a second meaning. The architecture must still answer queries efficiently. It must now also govern <strong>how much work an agent is permitted to create</strong>.</p><h3>Modelling agentic load</h3><p>It is tempting to say that agents break the familiar “users multiplied by requests” model of load. They do not break it. They change what sits inside it.</p><p>The database work generated by one agent run is the sum of the operations issued by every tool call, in every step, by every agent involved in the run:</p><pre>W(run) = Σ over agents  Σ over steps  Σ over tool calls  D</pre><pre>where<br>  A = agents taking part in the run (including delegates)<br>  S = steps taken by each agent<br>  T = tool calls per step<br>  D = database operations per tool call</pre><p>Under simplifying independence assumptions, the expected work per run is roughly:</p><pre>E[W] ≈ E[A] × E[S] × E[T] × E[D]</pre><p>Total load is still human-initiated runs multiplied by work per run. The product looks familiar. What has changed is the behaviour of each term.</p><p>In a designed API, S, T and D are close to constant. In an agent, they are random variables with long right tails. Worse, they are correlated. When a dependency slows down, the agent retries, which adds steps, which adds retrieval, which adds tool calls. The terms rise together, at the same moment, across every run in flight.</p><p>That correlation is the single most important idea in this article. Traditional capacity planning assumes that bad requests are rare and independent. Agentic capacity planning has to assume that bad runs arrive in waves.</p><h3>Running the numbers</h3><p>Abstract formulas are easy to agree with and easy to ignore. So here is a worked example, with figures chosen to be realistic for a large public-sector or enterprise service platform.</p><p><strong>The normal day</strong></p><p>→ 1,000,000 registered users</p><p>→ 10% active on a given day, each making three agent-assisted requests: <strong>300,000 runs per day</strong></p><p>→ A typical run: 8 steps × 1.5 tool calls × 4 operations = <strong>48 operations</strong></p><p>→ Daily volume: 14.4 million operations, or about <strong>170 operations per second</strong> on average</p><p>→ With a 10× business-hours peak: about <strong>1,700 operations per second</strong></p><p>That is comfortably within the reach of a modest, well-indexed replica set. If your intuition was that one million agent-assisted users is a hard MongoDB problem, the averages say otherwise.</p><p><strong>The bad twenty minutes</strong></p><p>Now introduce a 20-minute outage in a downstream service the agent calls during validation. Every run in that window retries until it hits its step ceiling. Suppose the ceiling is 60 steps:</p><p>→ Cost of an affected run: 60 × 1.5 × 4 = <strong>360 operations</strong>, a 7.5× increase</p><p>→ Load during peak: from about 1,700 to about <strong>12,500 operations per second</strong></p><p>→ A growing share of those operations are vector searches, which cost far more than indexed point reads</p><p>The cluster was sized for a normal day. The incident has made it seven times busier, and nothing about user behaviour changed.</p><p><strong>The day with no ceiling</strong></p><p>If the agent has no step ceiling at all, the second calculation has no upper bound. That is the 02:14 scenario.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*S7RvewucoXBu8emIXX7A7A.png" /><figcaption><em>Figure 2: database operations for a single agent run, with and without a ceiling.</em></figcaption></figure><p>The lesson from the numbers shapes everything that follows:</p><blockquote><strong><em>The scaling risk in agentic systems is not volume. It is unbounded, correlated volume.</em></strong></blockquote><p>A bigger cluster moves the point at which unbounded volume hurts you. It does not remove it.</p><h3>Five ways agents hurt a database</h3><p>Before looking at controls, it helps to name the failure modes. Most agentic incidents I would expect to see on a MongoDB platform fall into one of five patterns.</p><p><strong>1. The loop.</strong> The agent cannot reach sufficient confidence, so it keeps planning, retrieving and retrying. Each iteration looks reasonable in isolation. Together they look like a denial of service. This is the agentic equivalent of an infinite loop, except that no line of code is wrong.</p><p><strong>2. The fan-out.</strong> A supervisor agent delegates to five specialists, each of which calls three tools, each of which runs a search. Delegation multiplies load in a way that no individual agent’s design reveals. The amplification is visible only at the level of the whole run.</p><p><strong>3. Context bloat.</strong> Before every step, the agent reloads the whole conversation, every previous tool output and every possibly relevant memory. The longest runs, which are already the most expensive, get more expensive per step as they continue. Database reads and model tokens grow together.</p><p><strong>4. The leaky retrieval.</strong> Vector search returns the most semantically similar documents, and some of them belong to another tenant, have expired, or should never be shown to the requesting identity. Filtering after retrieval fixes correctness but wastes compute and quietly damages recall. Not filtering at all is a data protection incident.</p><p><strong>5. The confident bulk write.</strong> The agent identifies 5,000 records matching a condition and updates them all in seconds. The database handles it without complaint. The business discovers the next morning that the condition was wrong.</p><p>Notice that only one of the five (the loop) is primarily a capacity problem. The rest are correctness, security or governance problems that happen to surface in the database. That is why the controls that follow are not only about performance.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*noC24mG-xars9WgIBmt31A.png" /><figcaption><em>Figure 3: the shape of the platform the next eight sections build, piece by piece.</em></figcaption></figure><h3>Control 1: budgets enforced at the tool layer</h3><p>Every agent should be deployed with an explicit execution budget:</p><p>→ maximum steps, tool calls and delegated agents</p><p>→ maximum database operations and vector searches per run</p><p>→ maximum retrieved documents and context size</p><p>→ maximum retry depth per tool</p><p>→ maximum execution duration</p><p>→ maximum write scope and batch size</p><p>These are database protection controls, not AI settings. And they must be enforced outside the model.</p><p>A prompt instruction that says “stop after ten steps” is a suggestion. The model may follow it, or it may decide that step eleven is justified. A budget enforced at the tool layer is a control, because the model has no way to reach the database except through it.</p><p>The natural enforcement point is the tool gateway that sits between the agent runtime and MongoDB. That might be a custom service or an MCP server. The MongoDB MCP Server offers a read-only mode and lets individual tools be disabled, which is a sensible baseline for exploratory agents, but production budgets need per-run accounting.</p><p>A compact agent_runs document can hold both the budget and its consumption:</p><pre>{<br>  _id: &quot;run-12782&quot;,<br>  agentId: &quot;case-resolution-agent&quot;,<br>  agentVersion: &quot;3.4&quot;,<br>  tenantId: ObjectId(&quot;...&quot;),<br>  status: &quot;RUNNING&quot;,   // RUNNING | COMPLETED | BUDGET_EXCEEDED | ESCALATED<br>  budget: {<br>    maxSteps: 25,<br>    maxToolCalls: 40,<br>    maxDbOps: 300,<br>    maxVectorSearches: 20,<br>    maxDurationMs: 60000<br>  },<br>  usage: {<br>    steps: 0,<br>    toolCalls: 0,<br>    dbOps: 0,<br>    vectorSearches: 0<br>  },<br>  startedAt: ISODate(&quot;...&quot;),<br>  expiresAt: ISODate(&quot;...&quot;)<br>}</pre><p>Before each tool call, the gateway reserves the tool’s declared cost atomically. If the budget cannot cover it, the reservation fails and the tool never runs:</p><pre>// Reserve the declared cost of a tool call before executing it.<br>// The filter only matches if enough budget remains.<br>const run = await agentRuns.findOneAndUpdate(<br>  {<br>    _id: runId,<br>    status: &quot;RUNNING&quot;,<br>    &quot;usage.dbOps&quot;: { $lte: budget.maxDbOps - tool.declaredOps },<br>    &quot;usage.toolCalls&quot;: { $lt: budget.maxToolCalls }<br>  },<br>  {<br>    $inc: { &quot;usage.dbOps&quot;: tool.declaredOps, &quot;usage.toolCalls&quot;: 1 },<br>    $set: { lastActivityAt: new Date() }<br>  },<br>  { returnDocument: &quot;after&quot; }<br>);</pre><pre>if (!run) {<br>  await agentRuns.updateOne(<br>    { _id: runId, status: &quot;RUNNING&quot; },<br>    { $set: { status: &quot;BUDGET_EXCEEDED&quot;, endedAt: new Date() } }<br>  );<br>  throw new BudgetExceededError(runId, tool.name);<br>}</pre><p>Two design choices are worth calling out.</p><p>First, the reservation happens <strong>per tool call</strong>, using a declared cost, rather than per database operation. That keeps the accounting overhead to one small write per call. Compared with a model call that takes hundreds of milliseconds or several seconds, this cost is negligible. Tools that routinely exceed their declared cost should be flagged in telemetry and their declarations corrected.</p><p>Second, a run that hits its ceiling <strong>ends in a defined state</strong>. BUDGET_EXCEEDED is not an error to be hidden. It is a business event: the case gets escalated to a person, and the rate of budget exhaustion becomes one of the most useful health signals on the platform.</p><h3>Control 2: treat agent state as a portfolio</h3><p>In a conventional MongoDB application, the working set is mostly business documents and the index pages that serve them.</p><p>In an agentic platform, the working set also includes what I would call cognitive state: execution plans, conversation history, short-term and long-term memory, tool traces, checkpoints for resumable runs, embeddings and evaluation records.</p><p>These objects have different lifecycles, different consistency needs and different access patterns. Storing them all in one generalised “agent memory” collection gives you a flexible prototype and an expensive platform.</p><blockquote><strong><em>Agent memory is not one data type. It is a portfolio of states, and each state deserves its own model.</em></strong></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LBUV_BgDrE-Glv2H_ortyg.png" /><figcaption><em>Figure 4: seven kinds of state, seven different treatments in MongoDB.</em></figcaption></figure><p>Here is how I would treat each one.</p><p><strong>Execution state.</strong> Needs fast continuation and recovery. A small document per run, updated frequently, with a TTL index so it expires on its own once its retention window passes. Write concern w: 1 is usually acceptable, because a lost checkpoint means a restarted run, not a corrupted business record.</p><p><strong>Conversation state.</strong> Needs context reconstruction. Use the subset pattern: the most recent turns stored verbatim, older turns replaced by rolling summaries, with the full transcript held elsewhere if it must be retained.</p><p><strong>Long-term memory.</strong> Needs selective retrieval. Curated documents with relevance scores, embeddings and the metadata needed to filter them. Memory is written deliberately, not appended automatically.</p><p><strong>Tool history.</strong> Needs traceability and debugging. Written once, queried by time and run, rarely read after completion. This is exactly the profile time series collections are built for.</p><p><strong>Business state.</strong> Needs correctness. Domain-owned collections with w: &quot;majority&quot;, and multi-document transactions where an invariant spans documents. Agents act on this state. They do not own it.</p><p><strong>Knowledge embeddings.</strong> Need semantic retrieval. Versioned by embedding model and by source, indexed with the filter fields that enforce access and validity.</p><p><strong>Audit evidence.</strong> Needs immutability. An append-only collection, written by a role that has insert but no update or delete privileges, and read by a small number of controlled roles.</p><p>Two short examples make this concrete. Execution state that cleans up after itself:</p><pre>db.agent_runs.createIndex({ expiresAt: 1 }, { expireAfterSeconds: 0 });</pre><p>Tool history in a time series collection that ages out after ninety days:</p><pre>db.createCollection(&quot;tool_calls&quot;, {<br>  timeseries: {<br>    timeField: &quot;occurredAt&quot;,<br>    metaField: &quot;meta&quot;,          // { agentId, agentVersion, tenantId, toolName }<br>    granularity: &quot;seconds&quot;<br>  },<br>  expireAfterSeconds: 60 * 60 * 24 * 90<br>});</pre><p>There is a quieter benefit to this separation. It shapes what occupies the WiredTiger cache. Tool traces that are written once and seldom read should not compete for cache residency with long-term memory and business documents that agents and humans read constantly. When everything lives in one collection with one set of indexes, you lose the ability to make that distinction.</p><h3>Control 3: compile context, do not accumulate it</h3><p>A naïve agent treats context as a pile. Before each step, it loads the whole conversation, the user profile, every previous tool output and every memory that might be relevant, and sends all of it to the model in the hope that the model will pick out what matters.</p><p>That approach fails in two ways at once. It inflates model token consumption, and it creates database read amplification. Because each step reloads everything, the cost of a run grows faster than its length.</p><p>The alternative is to treat context as something compiled for each step by a dedicated service:</p><pre>Intent<br>  → Context policy        (what kinds of information this goal needs)<br>  → Targeted retrieval    (only those records, within access boundaries)<br>  → Assembly              (verbatim where required, summarised elsewhere)<br>  → Agent step</pre><p>For each step, the context service decides:</p><p>→ which facts are relevant to the current goal</p><p>→ which history can be replaced by a summary</p><p>→ which records must remain verbatim, such as the exact wording of a regulation</p><p>→ which records can be fetched on demand instead of preloaded</p><p>→ which items must be excluded because they are stale or unauthorised</p><p>→ which context version informed a decision, so the decision can be explained later</p><p>A simplified shape for the service looks like this:</p><pre>async function compileContext(run, goal) {<br>  const policy = contextPolicies.for(run.agentId, goal.type);</pre><pre>  const [profile, caseState, memories, knowledge] = await Promise.all([<br>    policy.needsProfile ? profiles.findOne(<br>      { _id: run.subjectId, tenantId: run.tenantId },<br>      { projection: policy.profileFields, comment: run.opContext }<br>    ) : null,<br>    cases.findOne(<br>      { _id: goal.caseId, tenantId: run.tenantId },<br>      { projection: policy.caseFields, comment: run.opContext }<br>    ),<br>    retrieveMemories(run, goal, policy.maxMemories),<br>    retrieveKnowledge(run, goal, policy.maxKnowledgeChunks)<br>  ]);</pre><pre>  return assemble({<br>    profile,<br>    caseState,<br>    memories,<br>    knowledge,<br>    history: await conversationSummary(run.sessionId),<br>    contextVersion: policy.version<br>  });<br>}</pre><p>The detail that matters is the projection. Retrieving only the fields a goal needs keeps documents small on the wire and in the model’s context, and it is a direct reduction in the work the database does per step.</p><p>The context service is also the best place to observe agent behaviour. Measure retrieval count per step, context size, cache hit ratio and, above all, <strong>repeated retrieval of the same document within a run</strong>. An agent that fetches the same case five times in one run is almost always looping. That metric will often warn you before CPU does.</p><h3>Control 4: give vector search deterministic boundaries</h3><p>Semantic retrieval changes how records are found. It does not remove the need for query design.</p><p>The most similar document is useless, or harmful, if it belongs to another tenant, has been superseded, is not yet in force, or cannot lawfully be shown to the requesting identity. In an enterprise setting, the real query is never “find documents similar to this prompt”. It is:</p><pre>Find semantically relevant documents<br>  within this tenant,<br>  accessible to this identity,<br>  valid for this jurisdiction,<br>  in force at this time,<br>  from approved sources,<br>  using the current embedding version.</pre><p>In MongoDB, those boundaries belong inside the vector search itself, as pre-filters. Every field you want to filter on has to be declared as a filter field in the vector search index definition:</p><pre>{<br>  &quot;fields&quot;: [<br>    { &quot;type&quot;: &quot;vector&quot;, &quot;path&quot;: &quot;embedding&quot;, &quot;numDimensions&quot;: 1536, &quot;similarity&quot;: &quot;cosine&quot; },<br>    { &quot;type&quot;: &quot;filter&quot;, &quot;path&quot;: &quot;tenantId&quot; },<br>    { &quot;type&quot;: &quot;filter&quot;, &quot;path&quot;: &quot;jurisdiction&quot; },<br>    { &quot;type&quot;: &quot;filter&quot;, &quot;path&quot;: &quot;status&quot; },<br>    { &quot;type&quot;: &quot;filter&quot;, &quot;path&quot;: &quot;effectiveFrom&quot; },<br>    { &quot;type&quot;: &quot;filter&quot;, &quot;path&quot;: &quot;embeddingVersion&quot; },<br>    { &quot;type&quot;: &quot;filter&quot;, &quot;path&quot;: &quot;accessGroups&quot; }<br>  ]<br>}</pre><p>The query then applies those boundaries during the approximate nearest neighbour search, not after it:</p><pre>db.knowledge_chunks.aggregate(<br>  [<br>    {<br>      $vectorSearch: {<br>        index: &quot;knowledge_vector_idx&quot;,<br>        path: &quot;embedding&quot;,<br>        queryVector: promptEmbedding,<br>        numCandidates: 200,<br>        limit: 10,<br>        filter: {<br>          $and: [<br>            { tenantId: ctx.tenantId },<br>            { jurisdiction: ctx.jurisdiction },<br>            { status: &quot;PUBLISHED&quot; },<br>            { effectiveFrom: { $lte: new Date() } },<br>            { embeddingVersion: &quot;text-emb-v3&quot; },<br>            { accessGroups: { $in: ctx.identity.groups } }<br>          ]<br>        }<br>      }<br>    },<br>    {<br>      $project: {<br>        text: 1,<br>        sourceId: 1,<br>        sourceVersion: 1,<br>        score: { $meta: &quot;vectorSearchScore&quot; }<br>      }<br>    }<br>  ],<br>  { comment: ctx.opContext }<br>);</pre><p>The difference between pre-filtering and post-filtering is not a stylistic preference. Filtering after retrieval can return fewer results than requested when most candidates are excluded, which quietly degrades answer quality. It also spends vector computation on documents the caller could never see. Filtering inside $vectorSearch keeps recall meaningful and keeps cost proportional to what the identity can actually access.</p><p>Three further practices matter at scale.</p><p><strong>Tune </strong><strong>numCandidates against a recall benchmark.</strong> A common starting point is 10 to 20 times the limit. Higher values improve recall at a direct cost in latency and compute. The right value is the lowest one that meets your recall target on a representative evaluation set, not the highest one that meets your latency target.</p><p><strong>Version embeddings explicitly.</strong> When you change embedding models, write the new vectors alongside the old ones under a new embeddingVersion, backfill, evaluate, then switch the filter. Rewriting in place leaves you with a period where half your corpus is searchable and half is noise.</p><p><strong>Resist the urge to vectorise everything.</strong> Business records, execution state and telemetry have different access patterns from knowledge. Embeddings belong with knowledge and curated memory. An agent reading a case by its identifier needs an index on _id, not a similarity search.</p><p>This is also where the document model earns its place. Because embeddings live in the same document as the metadata that governs them, there is no second store to keep in sync and no second security model to reconcile. Semantic discovery stays connected to business context by construction.</p><h3>Control 5: carry identity all the way to the data</h3><p>Once agents can act, the question “who changed this record?” gets harder to answer.</p><p>A database operation might be performed by a person directly, by an agent acting for a person, by a system agent acting for a process, by a specialist agent acting on delegation from a supervisor, or by a scheduled agent acting on its own. A write attributed to agent-service answers none of those questions. It is not an audit record. It is the absence of one.</p><p>Each state change made by an agent should carry the full chain of authority:</p><pre>{<br>  action: &quot;CASE_STATUS_UPDATED&quot;,<br>  entityId: ObjectId(&quot;...&quot;),<br>  tenantId: ObjectId(&quot;...&quot;),<br>  actor: {<br>    type: &quot;AGENT&quot;,<br>    agentId: &quot;case-resolution-agent&quot;,<br>    agentVersion: &quot;3.4&quot;,<br>    modelVersion: &quot;model-2026-06&quot;,<br>    policyVersion: &quot;case-policy-12&quot;<br>  },<br>  authority: {<br>    initiatedBy: &quot;user-7821&quot;,<br>    delegatedScope: &quot;case:update&quot;,<br>    mode: &quot;APPROVED&quot;,            // PROPOSED | APPROVED | AUTONOMOUS<br>    approvalId: &quot;approval-9182&quot;<br>  },<br>  execution: {<br>    runId: &quot;run-12782&quot;,<br>    toolCallId: &quot;tool-881&quot;,<br>    correlationId: &quot;journey-551&quot;<br>  },<br>  occurredAt: ISODate(&quot;...&quot;)<br>}</pre><p>This matters far beyond compliance. If an agent version makes an incorrect update across thousands of records, you need to identify exactly which agent version, model, policy and delegated authority produced it, and you need to find every record it touched. Without that chain, rollback becomes guesswork.</p><p>Three MongoDB details make identity work in practice.</p><p><strong>Write the actor into the business document.</strong> Change streams tell you what changed. They do not tell you who changed it. If the acting agent and its authority are recorded in the document itself, for example in a lastModifiedBy sub-document, a change stream consumer can build a reliable audit trail. If they are not, no downstream process can recover them.</p><p><strong>Attach execution context to every operation with </strong><strong>comment.</strong> Every operation issued through the tool gateway should carry the run context:</p><pre>await cases.updateOne(filter, update, {<br>  comment: {<br>    runId: ctx.runId,<br>    agentId: ctx.agentId,<br>    agentVersion: ctx.agentVersion,<br>    toolCallId: ctx.toolCallId<br>  }<br>});</pre><p>The comment appears in the slow query log, the database profiler and $currentOp. At 02:14, this is the difference between &quot;something is hammering the cases collection&quot; and &quot;version 3.4 of the case-resolution agent, run 12782, tool call 881, is hammering the cases collection&quot;.</p><p><strong>Scope database roles by agent class.</strong> A read-only research agent and a case-resolution agent with delegated write authority should not share credentials. Custom roles with the minimum privileges for each class mean that a compromised or misbehaving agent is limited by the database, not only by the application.</p><blockquote><strong><em>Agents should never appear to MongoDB as anonymous backend traffic.</em></strong></blockquote><h3>Control 6: separate reading from committing</h3><p>Agents are naturally read-heavy. They gather context, compare history, inspect policy and collect evidence before deciding. That is where they add most of their value, and read load is comparatively easy to scale.</p><p>The risk profile changes sharply when agents can write.</p><p>A person updates one case after reviewing one screen. An agent can identify 5,000 cases matching a condition and update all of them before anyone notices. The question is no longer whether MongoDB can absorb the writes. It can. The question is whether the business should allow that much change to happen without a control boundary.</p><p>Tools should expose distinct capabilities rather than a single “update” verb:</p><p>→ <strong>Read:</strong> retrieve state</p><p>→ <strong>Propose:</strong> record an intended change with its evidence</p><p>→ <strong>Simulate:</strong> compute the effect of a change without applying it</p><p>→ <strong>Validate:</strong> check a proposal against deterministic policy</p><p>→ <strong>Execute:</strong> apply a single approved change</p><p>→ <strong>Execute in bulk:</strong> apply an approved change to a bounded set</p><p>→ <strong>Reverse:</strong> undo a change using its recorded prior state</p><p>→ <strong>Escalate:</strong> hand the decision to a person</p><p>The architectural principle is simple:</p><pre>Agent reasoning<br>  → Proposed action<br>  → Deterministic policy gate<br>  → Authorised command<br>  → MongoDB state transition</pre><p>The model may propose. A deterministic gate decides whether the proposal becomes a command. MongoDB remains the system of record, and the agent remains a probabilistic participant governed by rules it cannot rewrite.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yMMtANoetbnIOvKHW_76kw.png" /><figcaption><em>Figure 5: a model may propose a change, a deterministic gate decides whether it is committed.</em></figcaption></figure><p>In MongoDB terms, that translates into five practices.</p><p><strong>Proposals are data.</strong> Proposed actions are written to an action_proposals collection with their evidence and the context version that produced them, then approved by a person or a policy engine. This also gives you a complete record of what agents wanted to do, which is invaluable for evaluation.</p><p><strong>Commands are idempotent.</strong> Every command carries an idempotency key backed by a unique index. A retried tool call, and agents retry constantly, cannot apply the same change twice:</p><pre>db.commands.createIndex({ idempotencyKey: 1 }, { unique: true });</pre><p><strong>Invariants use transactions.</strong> Where a state transition must update several documents together, such as a case, its assignment and its audit entry, use a multi-document transaction with w: &quot;majority&quot;.</p><p><strong>Bulk scope is capped.</strong> The tool gateway enforces maximum batch sizes. Anything above a threshold requires a dry run with before-and-after evidence and an explicit approval.</p><p><strong>Transitions are reversible.</strong> Every change records the prior value it replaced, so that the effects of a faulty agent version can be reversed by run, by version or by time window.</p><h3>Control 7: shard for the agents you will have</h3><p>Human traffic follows time zones, business hours and campaigns. You learn its shape after a few months in production.</p><p>Agent traffic adds triggers that have no human rhythm: scheduled runs, cascading reactions to events, delegation chains, reprocessing after a failure, memory consolidation, evaluation runs and re-indexing. And a new prompt version can change database demand overnight with no change in users at all. If an agent starts retrieving twenty records instead of five, or validates after every step instead of once at the end, load multiplies the moment the release goes out.</p><p>This has two consequences for MongoDB design.</p><p><strong>Shard keys must anticipate agent write patterns.</strong> Agent-generated collections tend to be keyed by run identifiers or timestamps, both of which increase monotonically. A monotonic shard key sends every new insert to the same chunk on the same shard, and bulk agent activity turns that into a hotspot. A compound key led by tenant, with a hashed component, spreads agent writes while keeping tenant-scoped queries targeted:</p><pre>sh.shardCollection(&quot;agents.agent_memory&quot;, { tenantId: 1, subjectId: &quot;hashed&quot; });</pre><p>Very large tenants still need watching, because a single tenant can become a hotspot on its own. Monitor data and operation distribution per tenant, not only per collection.</p><p><strong>Agent releases are workload releases.</strong> Before promoting a new agent version, replay a representative set of goals against it and compare it with the current version:</p><p>→ mean and p99 steps, tool calls and database operations per run</p><p>→ vector searches and retrieved documents per step</p><p>→ repeated retrieval within a run</p><p>→ write rate and batch sizes</p><p>→ behaviour when a dependency is slow or unavailable</p><p>→ cost per successful outcome</p><p>That fifth item is the one most teams skip, and it is the one that produces incidents like 02:14. Test the new version with a dependency deliberately failing. If p99 cost per run jumps by an order of magnitude, the release is not ready, however good its answers are.</p><p>Traditional load testing asks how many concurrent users a system can support. Agentic load testing also has to ask:</p><blockquote><strong><em>How much work can each user cause the system to generate, and what happens to that number when something fails?</em></strong></blockquote><h3>Control 8: isolate workloads by demand class</h3><p>Once agents are first-class participants, MongoDB capacity is consumed by four distinct classes of demand:</p><p>→ <strong>Human demand:</strong> direct user interactions</p><p>→ <strong>Application demand:</strong> APIs, workflows and transactional services</p><p>→ <strong>Background demand:</strong> integrations, analytics and synchronisation</p><p>→ <strong>Agentic demand:</strong> planning, retrieval, memory, tools, delegation and autonomous action</p><p>These classes should not draw on shared capacity without boundaries. A citizen-facing transaction should not slow down because an agent is consolidating memory. A payment write should not queue behind bulk re-embedding. A regulatory query should not compete on equal terms with an agent evaluating an optional recommendation.</p><p>Workload isolation is how the platform expresses business priority. MongoDB gives you several concrete mechanisms for it.</p><p><strong>Separate client pools.</strong> Give each demand class its own MongoClient, with its own pool size, timeout and application name:</p><pre>const transactional = new MongoClient(uri, {<br>  appName: &quot;txn-services&quot;,<br>  maxPoolSize: 200<br>});</pre><pre>const agentGateway = new MongoClient(uri, {<br>  appName: &quot;agent-gateway&quot;,<br>  maxPoolSize: 40,<br>  timeoutMS: 5000<br>});</pre><p>Capping the agent pool means agent concurrency cannot exhaust the connections that transactional services rely on. The appName shows up in logs and $currentOp, which adds a second, coarser layer of attribution alongside the per-operation comment.</p><p><strong>Operation time limits.</strong> Every agent-issued operation should carry a time limit, through maxTimeMS or the driver’s client-side timeout. MongoDB 8.0 also introduced a cluster-wide default time limit for read operations, which makes a useful backstop for anything that slips through.</p><p><strong>Node-level isolation.</strong> Route evaluation runs and analytical reads to dedicated analytics nodes using read preference tags:</p><pre>const evaluation = new MongoClient(uri, {<br>  appName: &quot;agent-evaluation&quot;,<br>  readPreference: &quot;secondary&quot;,<br>  readPreferenceTags: [{ nodeType: &quot;ANALYTICS&quot; }]<br>});</pre><p><strong>Search isolation.</strong> On Atlas, dedicated search nodes separate vector and full-text workloads from the operational database. A surge in semantic retrieval then consumes search node resources instead of the resources serving transactional reads and writes.</p><p><strong>Write concern by class.</strong> Business state uses w: &quot;majority&quot;. Tool telemetry and execution checkpoints can use w: 1 where losing a trace is acceptable. There is no reason to pay majority-acknowledgement latency for a debug record.</p><p><strong>A real kill switch.</strong> Because every agent operation carries a comment, operators can find a runaway agent’s operations directly:</p><pre>db.getSiblingDB(&quot;admin&quot;).aggregate([<br>  { $currentOp: { allUsers: true } },<br>  { $match: { &quot;command.comment.agentId&quot;: &quot;case-resolution-agent&quot; } },<br>  { $project: { opid: 1, secs_running: 1, ns: 1, &quot;command.comment&quot;: 1 } }<br>]);</pre><pre>// then, for each offending operation:<br>db.killOp(opid);</pre><p>Killing operations stops the current damage. Stopping the next wave is a job for the tool gateway: flip the agent’s status to suspended so that every new tool call is refused. And from MongoDB 8.0, persistent query settings can reject a specific query shape outright while the agent is being fixed:</p><pre>db.adminCommand({<br>  setQuerySettings: {<br>    find: &quot;cases&quot;,<br>    filter: { status: &quot;OPEN&quot;, region: &quot;north&quot; },   // shape is matched, literal values are not<br>    $db: &quot;operations&quot;<br>  },<br>  settings: { reject: true }<br>});</pre><p>Use that last one with care. It rejects the shape for every caller, humans included, so it is a tool for a genuine emergency rather than routine throttling.</p><h3>Measuring what matters: the outcome</h3><p>Requests per second lose most of their meaning when one request can run an agent for thirty seconds and trigger dozens of downstream operations. The useful unit of capacity is the completed business outcome.</p><p>Report the three kinds of work separately, each per successful outcome:</p><pre>DB operations per outcome  = database operations / successful outcomes<br>Tokens per outcome         = model tokens / successful outcomes<br>Tool calls per outcome     = tool calls / successful outcomes</pre><p>Combine them only once they share a unit, which in practice means cost:</p><pre>Cost per outcome = (database cost + model cost + tool cost) / successful outcomes</pre><p>Useful operational measures built on this idea include:</p><p>→ database reads per resolved case</p><p>→ vector searches per accepted answer</p><p>→ writes per completed onboarding</p><p>→ human escalations per 1,000 runs</p><p>→ share of runs ending at a budget ceiling</p><p>→ repeated retrievals per run</p><p>The distinction this draws is between useful scale and mere activity. An agent that completes 1,000 outcomes with 10,000 database operations is a better system than one that completes the same outcomes with 100,000, even if both meet their latency targets. Latency tells you the platform is coping. Outcome efficiency tells you whether it should have to.</p><p>It also gives you the language to talk to the business. “The cluster is at 60% CPU” means little to a programme director. “Resolving a case now costs 40% less database work than last quarter, and escalations have dropped by a third” means a great deal.</p><h3>Back to 02:14</h3><p>Replay the opening scenario with the controls in place.</p><p>The document validation service starts failing at 01:34. The case-resolution agent retries, exactly as before. But each run is now reserving budget before every tool call. Within a few minutes, affected runs begin reaching their ceiling of 300 database operations. Each one ends in BUDGET_EXCEEDED, writes a clean final state and escalates the case to the morning queue.</p><p>The worst case is now bounded. At most 3,000 runs at 300 operations each: under a million operations spread across forty minutes, on an agent connection pool capped at 40 connections. The transactional pool is untouched. Vector searches are running on dedicated search nodes. Citizen-facing latency does not move.</p><p>At 01:41, an alert fires. Not on CPU or latency, but on outcome health: the share of runs ending at a budget ceiling has crossed 5%. The alert names the agent and its version, because every operation carries that in its comment. The on-call engineer sees at a glance that runs are exhausting budget on the validation tool, checks the dependency, and suspends the agent at the gateway until the service recovers.</p><p>Nobody is paged at 02:14. The morning team finds a few hundred escalated cases, each with a complete record of what the agent tried, and a precise account of why it stopped.</p><p>That is what good agentic architecture looks like in practice. Not an agent that never fails, but a platform where failure is bounded, attributable and reversible.</p><h3>Objections worth answering</h3><p><strong>“Why not just scale the cluster?”</strong></p><p>Because the problem is unbounded, correlated demand, and no fixed amount of capacity bounds it. A larger cluster raises the threshold at which a looping agent hurts you, and it raises your costs every hour of every normal day in order to do so. Budgets cost almost nothing on a normal day and cap the damage on a bad one.</p><p><strong>“Shouldn’t embeddings live in a dedicated vector database?”</strong></p><p>It depends on what you are optimising for. A specialised vector store may offer capabilities that matter for some workloads. The trade-off is a second system to keep in sync with the metadata that governs access and validity, and a second security model to keep consistent with the first. For enterprise retrieval, where tenant, access, jurisdiction and effective-date filters are not optional, keeping embeddings in the same document as that metadata removes an entire class of synchronisation and leakage problems. Make the decision deliberately rather than by default.</p><p><strong>“Isn’t this the orchestration framework’s job?”</strong></p><p>Orchestration frameworks decide what an agent does next. They should not be the only thing standing between an agent and your operational database. The framework may be upgraded, replaced or misconfigured. Controls enforced at the tool gateway and in the database, such as budgets, scoped roles, idempotency keys and time limits, hold regardless of which framework is driving the agent.</p><p><strong>“Won’t all this governance slow agents down?”</strong></p><p>A budget reservation is one small indexed write per tool call. A comment adds a few bytes to each operation. Projections and pre-filters reduce work. Against model calls measured in hundreds of milliseconds or seconds, the overhead is negligible, and several of these controls make agents faster by reducing what they retrieve.</p><h3>Where to start</h3><p>If you are running agents against MongoDB today and have none of this in place, the order matters. Start with the controls that bound damage, then the ones that explain it, then the ones that optimise it.</p><p><strong>First, bound.</strong> Put a tool gateway between agents and the database, enforce per-run budgets, cap the agent connection pool and set time limits on every agent operation. This alone would have prevented the 02:14 incident.</p><p><strong>Second, attribute.</strong> Add the comment convention to every agent operation, write actor and authority into business documents, and give each agent class its own database role. From this point, every incident is explainable.</p><p><strong>Third, protect.</strong> Move vector retrieval to pre-filtered queries, and route every agent write through propose, validate and execute stages with idempotency keys and batch limits.</p><p><strong>Fourth, optimise.</strong> Split agent state into purpose-specific collections, compile context per step, isolate demand classes onto the right nodes and start reporting cost per outcome.</p><p>Each stage makes the next one easier, and the first stage can usually be delivered in weeks rather than months.</p><h3>Readiness checklist</h3><p>→ Every agent has an execution budget enforced at the tool gateway, not in the prompt</p><p>→ Runs that exhaust their budget end in a defined state and escalate</p><p>→ Agent state is split by purpose, using TTL indexes and time series collections where they fit</p><p>→ Context is compiled per step, with projections, and repeated retrieval is measured</p><p>→ Vector search uses pre-filters for tenant, access, status, effective date and embedding version</p><p>→ Agent identity and authority are written into business documents</p><p>→ Every agent operation carries a comment with run context</p><p>→ Each agent class authenticates with its own scoped database role</p><p>→ Writes pass through propose, validate and execute stages, with idempotency keys and batch limits</p><p>→ Every state change records its prior value so it can be reversed</p><p>→ Shard keys for agent-generated data avoid monotonic hotspots</p><p>→ Demand classes have separate connection pools, time limits and, where needed, dedicated nodes</p><p>→ New agent versions are tested on p99 cost and dependency-failure behaviour before release</p><p>→ Capacity and cost are reported per successful business outcome</p><h3>Conclusion</h3><p>Agents do not change what MongoDB is good at. They change what a platform built on it has to be responsible for.</p><p>For most of MongoDB’s history, scaling has meant distributing data and work across machines: better indexes, better shard keys, more nodes in the right places. That work still matters, and every control in this article depends on it. But agents add a second dimension. The demand arriving at the database is no longer designed by an engineer and paced by a person. It is chosen at runtime by software that can loop, delegate, retry and act on its own.</p><p>The worked example makes the point plainly. On a normal day, a million agent-assisted users produce load that a modest cluster handles comfortably. On a bad day, the same users produce seven times that load, or an unlimited amount, without a single person behaving any differently. The difference between those two outcomes is not hardware. It is whether the architecture decides how much work an agent is allowed to create.</p><p>That is why the controls in this article belong to the platform and not to the prompt. Budgets at the tool gateway bound what a run can cost. Purpose-specific state models and compiled context keep the working set lean. Pre-filtered vector search keeps retrieval both relevant and lawful. Identity written into the data, and carried on every operation, makes every action explainable. Gated, idempotent and reversible writes keep agents as participants in business change rather than owners of it. Workload isolation ensures that when something does go wrong, it goes wrong in the agent’s lane and not in the citizen’s.</p><p>Few of these ideas are exotic. Most rely on capabilities MongoDB has offered for years: TTL indexes, time series collections, transactions, unique indexes, read preference tags, operation comments and scoped roles. What is new is the discipline of applying them together, deliberately, to a class of user that did not exist before.</p><p>At scale, the most dangerous agent is not the one that makes a wrong decision. It is the one that can create unlimited work, read unlimited context and take unlimited action, while the architecture around it cannot explain, constrain or reverse what it did.</p><p><strong>So the question for architects has moved on. It is no longer whether MongoDB can support one million users. It can. The question is whether the platform can explain, constrain and reverse the work that one million users, and the agents acting for them, are allowed to create.</strong></p><p>At 02:14, the answer to that question is the difference between a page and a quiet night.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ead9ffd2fe22" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[ Building Systems That Know Why They Exist ~ When Data, Logic, and Intent Finally Align]]></title>
            <link>https://medium.com/@eliehannouch800/building-systems-that-know-why-they-exist-when-data-logic-and-intent-finally-align-294810cf4fa5?source=rss-79f152ec915------2</link>
            <guid isPermaLink="false">https://medium.com/p/294810cf4fa5</guid>
            <category><![CDATA[genai]]></category>
            <category><![CDATA[rac]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[mongodb]]></category>
            <category><![CDATA[requirementascode]]></category>
            <dc:creator><![CDATA[Elie hannouch]]></dc:creator>
            <pubDate>Wed, 15 Oct 2025 19:07:43 GMT</pubDate>
            <atom:updated>2025-10-15T19:07:43.156Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fKlXRaOCelfpXA_XhNaitA.png" /></figure><p>Every software system begins with intent. A human decision. A statement of what should exist, how it should behave, and why it matters. But somewhere along the way, that intent dies. It’s decomposed into documentation, user stories, and scattered logic. It becomes a shadow of itself fragmented across layers of code, processes, and people who no longer remember what it was meant to achieve.</p><p>We’ve automated infrastructure. We’ve scripted configurations. We’ve even made AI write code for us. But we’ve never automated <strong>understanding</strong>. Our systems execute brilliantly yet comprehend nothing. They operate without memory of purpose. They run, but they do not know <em>why</em>.</p><p>This is the gap that <strong>Requirements-as-Code (RaC)</strong> exists to close. It’s not a tool or a framework. It’s an engineering discipline an architectural shift that treats requirements, business rules, and behavioral constraints as <strong>executable data models</strong>, not static documentation. Under RaC, requirements don’t describe software they <em>are</em> the software. They exist as structured data, versioned, queryable, interpretable at runtime, and observable across the system lifecycle.</p><p>In a RaC environment, logic isn’t buried inside code. It’s stored as intent. The system no longer executes a developer’s assumption; it interprets a documented purpose. Each requirement becomes a living object linked to context, constraints, and dependencies. Together they form a semantic graph of meaning, a cognitive skeleton that defines what the system knows about itself.</p><p>MongoDB becomes the natural substrate for this idea. Its document model mirrors the structure of thought nested, relational, self-descriptive. Each requirement can be stored as a document; relationships emerge through references and embedded structures; versioning and lineage form the backbone of traceability. Change Streams act as the system’s nervous system, broadcasting intent mutations as real-time events. Vector Search provides semantic proximity linking related requirements and behaviors into conceptual neighborhoods. Queryable Encryption ensures that comprehension does not compromise confidentiality. In this architecture, MongoDB stops being a persistence layer. It becomes a <strong>substrate of cognition,</strong> the place where data, logic, and intent converge.</p><p>A RaC system operates as a semantic control loop: specification, interpretation, execution, observation, adaptation. The specification is defined as data. The interpretation layer compiles it into runtime behavior. Execution consumes it. Observation compares reality to intent. Adaptation feeds learning back into the model. The system, in effect, learns to regulate itself not only against functional expectations, but against purpose.</p><p>Generative AI made this even more necessary. When models can generate logic, schemas, and workflows, we gain speed but lose semantic safety. AI can produce code that functions yet fails to represent intent. RaC restores the missing boundary. It defines the <strong>contract of meaning</strong> between human language and machine behavior. AI becomes a companion process: a compiler of natural language into structured intent, while the RaC layer becomes the governor that validates and constrains it. Together, they enable systems that not only generate faster, but reason more safely.</p><p>The implications for engineering are profound. Behavior becomes data-driven instead of hardcoded. New rules can be introduced like new documents, not deployments. Traceability becomes intrinsic, every action traceable to its originating requirement. Failure becomes explainable, the system can articulate <em>why</em> something happened, not just that it did. Refactoring becomes evolutionary driven by metadata and intent, not arbitrary branching. Governance becomes continuously embedded in the same substrate that executes logic.</p><p>RaC is not a reinvention of programming. It’s a restoration of alignment between human thought and system behavior. It treats software not as an accumulation of code, but as a network of meaning. Each requirement becomes a neuron; each dependency, a synapse; each vector, a trace of memory. MongoDB provides the cognitive architecture; AI becomes the reasoning cortex; RaC is the connective tissue that allows the system to think coherently.</p><p>This evolution isn’t about speed. It’s about fidelity. It’s about reducing semantic drift the slow entropy that occurs when software stops reflecting its original design purpose. In traditional systems, requirements die after implementation. In RaC systems, they live as part of the runtime, observable, testable, and correctable. The system carries its intent forward, continuously validating that what it does remains aligned with what it was meant to do.</p><p>We often say automation made systems faster. But RaC makes them <strong>aware</strong>. It transforms architecture into dialogue. It closes the gap between what we build and what we meant to build. It’s the bridge between code and conscience, the point where a system stops merely executing, and starts reasoning.</p><p>Infrastructure-as-Code automated machines. Requirements-as-Code automates understanding. MongoDB gives it structure; AI gives it language; and together they bring us closer to a new kind of system one that doesn’t just function, but <em>knows why it functions.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=294810cf4fa5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How MongoDB Decides What to Forget ?]]></title>
            <link>https://medium.com/@eliehannouch800/how-mongodb-decides-what-to-forget-5ee7bbaed142?source=rss-79f152ec915------2</link>
            <guid isPermaLink="false">https://medium.com/p/5ee7bbaed142</guid>
            <category><![CDATA[database]]></category>
            <category><![CDATA[mongodb]]></category>
            <category><![CDATA[data-driven]]></category>
            <category><![CDATA[nosql-database]]></category>
            <dc:creator><![CDATA[Elie hannouch]]></dc:creator>
            <pubDate>Tue, 14 Oct 2025 17:27:28 GMT</pubDate>
            <atom:updated>2025-10-14T17:27:28.735Z</atom:updated>
            <content:encoded><![CDATA[<h3>How MongoDB Decides What to Forget ?</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2RyMETT6diUSVGKdJPoQgQ.png" /></figure><p>Inside MongoDB’s storage engine, WiredTiger, <strong>nothing happens by accident.</strong></p><p>Every page in memory exists under policy — governed, measured, and continuously evaluated against the limits of RAM, I/O bandwidth, and checkpoint cadence. Eviction is not cleanup. It’s <strong>runtime arbitration between volatility and durability</strong>.</p><p>When the process starts, WiredTiger allocates a fixed memory region known as the <strong>cache arena</strong> typically 50% of physical RAM. Within that space live <strong>B-tree pages</strong>: internal nodes, leaf nodes, and history-store entries. Each page carries operational metadata: <strong>dirty, clean, hazard-protected, in-use, last_access_time, and generation</strong>. This metadata feeds into a <strong>per-page score</strong>, which informs the eviction subsystem’s next decision.</p><p>At steady state, the cache holds both <strong>clean pages</strong> (byte-for-byte identical to disk) and <strong>dirty pages</strong> (modified since last checkpoint). When total occupancy approaches the configured <strong>eviction_target</strong>, a <strong>cooperative multi-threaded subsystem</strong> engages background <strong>evict_lru_worker threads</strong> start scanning the B-trees, guided by internal structures like WT_PAGE_INDEX and WT_REF, using an adaptive <strong>LRU-like algorithm</strong> tuned by recent access histograms.</p><p>The core loop looks deceptively simple:</p><h4><strong>Scan → Score → Hazard Check → Write (if dirty) → Evict → Adjust Metrics</strong></h4><p>But each stage involves subtle synchronization:</p><ul><li><strong>Scanning</strong> is probabilistic workers don’t traverse full trees; they sample subtrees to avoid lock contention.</li><li><strong>Scoring</strong> blends recency and reuse. WiredTiger doesn’t track strict LRU; it uses <em>generational counters</em> that decay over time.</li><li><strong>Hazard checking</strong> ensures no session currently holds a read cursor (WT_SESSION_IMPL) referencing that page.</li><li><strong>Write phase</strong> invokes wt_evict_dirty_leaf(), writing through the <strong>WiredTiger Journal (WAL)</strong> if necessary, updating the <strong>durable history store</strong> when timestamped versions are needed.</li><li><strong>Evict phase</strong> frees the WT_PAGE structure and updates eviction statistics for adaptive tuning.</li></ul><p>The eviction engine constantly measures its own efficiency using internal metrics like eviction server candidate queue length, eviction active workers, and eviction empty queue passes.<br> When efficiency drops meaning eviction cannot keep up with mutation rate the engine escalates through defined phases:</p><ol><li><strong>Passive eviction</strong> — background threads maintain equilibrium below eviction_target.</li><li><strong>Aggressive eviction</strong> — triggered near eviction_trigger; application threads begin assisting.</li><li><strong>Emergency eviction</strong> — above 95% occupancy; eviction threads run synchronously, blocking writers until space is reclaimed.</li></ol><p>This escalation is controlled by the <strong>Eviction Server</strong>, which coordinates worker pools using an internal <strong>task queue</strong> implemented with spinlocks and condition variables to minimize scheduler overhead.</p><p>Eviction is deeply coupled to <strong>checkpointing</strong>. While checkpoints provide stable on-disk consistency by writing all dirty pages up to a <strong>stable timestamp</strong>, eviction continuously frees memory between checkpoints. If eviction outpaces checkpointing, dirty data accumulates in the <strong>WiredTiger Journal (WiredTigerLog.*)</strong>. If checkpointing lags, the <strong>tracked dirty bytes</strong> metric grows, triggering <strong>flow control</strong>.</p><p>Flow control, implemented in the replication coordinator layer, slows down writers at the client operation level. It computes a delay token proportional to (dirty_bytes / cache_size) and (replication_lag / apply_rate). In effect, <strong>the cache backpressure propagates up to the client layer</strong> a closed feedback loop spanning the storage, replication, and client I/O pipelines.</p><p>A key subtlety lies in how WiredTiger separates <strong>internal</strong> and <strong>leaf page eviction quotas</strong>. Internal nodes are critical for cursor traversal; evicting too many increases read amplification. Thus, the system enforces ratios like <strong>internal_pages_evicted / leaf_pages_evicted ≈ 0.05–0.1</strong> to maintain navigational efficiency. This is continuously adjusted based on statistics in <strong>WiredTigerStatType::cache_eviction_internal.</strong></p><p>Another often-missed detail: <strong>dirty pages are written incrementally, not atomically</strong>. WiredTiger splits them into smaller “update chains,” writing partial updates if page size exceeds split_page_max. This design avoids long stalls under large document updates a crucial factor for workloads with oversized BSON documents or frequent $push operations.</p><p>In a high-throughput deployment, this entire mechanism behaves like a distributed feedback system:</p><ul><li>The <strong>sensor layer</strong> measures cache pressure (bytes currently in cache, dirty_bytes_ratio, eviction_trigger).</li><li>The <strong>controller layer</strong> adjusts eviction thread counts and page scoring heuristics.</li><li>The <strong>actuator layer</strong> executes writes, advancing checkpoints and freeing pages.</li><li>The <strong>stabilizer layer</strong> (flow control) modulates client throughput to keep the system near steady state.</li></ul><p>This is control theory implemented in a storage engine a practical PID loop embedded inside C code and shared locks.</p><p>The result is not perfection but <strong>bounded instability</strong>: cache utilization oscillates, dirty bytes rise and fall, eviction queues expand and drain. That’s the intended state. A flat metric line means eviction has stalled no breathing room left.</p><p>Engineers who monitor clusters at scale don’t watch “cache size”; they track oscillation frequency. A healthy system breathes roughly once per checkpoint interval. When the oscillation slows, latency spikes. When it quickens uncontrollably, disk I/O saturates. Balancing those two edges is what the eviction engine was designed to do.</p><p>What fascinates me most is how <strong>architecturally pure</strong> this mechanism is.<br> It doesn’t rely on heuristics alone; it encodes system health as measurable control variables. Memory, disk, and replication are not separate domains, they’re interconnected feedback loops sharing a common timebase. That’s what gives MongoDB its stability under unpredictable workloads.</p><p>Eviction isn’t a cleanup cycle. It’s a <strong>continuous act of governance</strong> the system regulating its own volatility so that higher layers can deliver consistency and performance. And the deeper you study it, the more you realize:</p><blockquote>The hardest part of building reliable systems isn’t writing data. <strong>It’s knowing exactly when to let go.</strong></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5ee7bbaed142" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[       : …]]></title>
            <link>https://medium.com/@eliehannouch800/-b71bfe534213?source=rss-79f152ec915------2</link>
            <guid isPermaLink="false">https://medium.com/p/b71bfe534213</guid>
            <category><![CDATA[enterprise-software]]></category>
            <category><![CDATA[enterprise-solutions]]></category>
            <category><![CDATA[architectural-design]]></category>
            <category><![CDATA[product]]></category>
            <dc:creator><![CDATA[Elie hannouch]]></dc:creator>
            <pubDate>Wed, 04 Jun 2025 20:07:03 GMT</pubDate>
            <atom:updated>2025-06-05T05:24:09.244Z</atom:updated>
            <content:encoded><![CDATA[<h3>𝗛𝗼𝘄 𝗘𝗻𝘁𝗲𝗿𝗽𝗿𝗶𝘀𝗲 𝗦𝘆𝘀𝘁𝗲𝗺𝘀 𝗔𝗰𝘁𝘂𝗮𝗹𝗹𝘆 𝗪𝗼𝗿𝗸 𝗮𝘁 𝗦𝗰𝗮𝗹𝗲 : 𝗪𝗵𝗲𝗻 𝗙𝗮𝗶𝗹𝘂𝗿𝗲 𝗶𝘀 𝗡𝗼𝘁 𝗮𝗻 𝗢𝗽𝘁𝗶𝗼𝗻 — The Sovereign Execution Model</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ExevFxoAs5OEf_8_bLUuUw.png" /></figure><p>In true enterprise-grade systems — financial networks, mission-critical platforms, regulated industries, national infrastructures — you are not building software. You are 𝗲𝗻𝗴𝗶𝗻𝗲𝗲𝗿𝗶𝗻𝗴 𝘀𝘆𝘀𝘁𝗲𝗺𝘀 𝘁𝗵𝗮𝘁 𝗴𝗼𝘃𝗲𝗿𝗻 𝗶𝗿𝗿𝗲𝘃𝗲𝗿𝘀𝗶𝗯𝗹𝗲 𝘀𝘁𝗮𝘁𝗲 𝘂𝗻𝗱𝗲𝗿 𝗰𝗼𝗺𝗽𝗹𝗲𝘅 𝗿𝘂𝗹𝗲𝘀, 𝗱𝗶𝘀𝘁𝗿𝗶𝗯𝘂𝘁𝗲𝗱 𝗮𝗰𝘁𝗼𝗿𝘀, 𝗮𝗻𝗱 𝗮𝗯𝘀𝗼𝗹𝘂𝘁𝗲 𝗮𝘂𝗱𝗶𝘁𝗮𝗯𝗶𝗹𝗶𝘁𝘆.</p><p>At scale, 𝗲𝘃𝗲𝗿𝘆 𝗿𝗲𝗮𝗹 𝘀𝘆𝘀𝘁𝗲𝗺 𝗰𝗼𝗹𝗹𝗮𝗽𝘀𝗲𝘀 𝘁𝗼 𝗼𝗻𝗲 𝘂𝗻𝗶𝘃𝗲𝗿𝘀𝗮𝗹, 𝗶𝗺𝗺𝘂𝘁𝗮𝗯𝗹𝗲 𝗲𝘅𝗲𝗰𝘂𝘁𝗶𝗼𝗻 𝗰𝗵𝗮𝗶𝗻:</p><p>A 𝗣𝗲𝗿𝘀𝗼𝗻𝗮 𝗶𝗻𝘁𝗲𝗿𝗮𝗰𝘁𝘀 𝘄𝗶𝘁𝗵 𝗮𝗻 𝗢𝗯𝗷𝗲𝗰𝘁 𝘁𝗵𝗿𝗼𝘂𝗴𝗵 𝘁𝗵𝗲 𝗨𝗜, 𝗽𝗲𝗿𝗳𝗼𝗿𝗺𝗶𝗻𝗴 𝗮𝗻 𝗔𝗰𝘁𝗶𝗼𝗻 that 𝗴𝗲𝗻𝗲𝗿𝗮𝘁𝗲𝘀 𝗮𝗻 𝗶𝗺𝗺𝘂𝘁𝗮𝗯𝗹𝗲 𝗘𝘃𝗲𝗻𝘁, which is 𝗽𝗿𝗼𝗰𝗲𝘀𝘀𝗲𝗱 𝗯𝘆 𝗮 𝘀𝘁𝗮𝘁𝗲𝗹𝗲𝘀𝘀 𝗛𝗮𝗻𝗱𝗹𝗲𝗿, 𝘁𝗿𝗶𝗴𝗴𝗲𝗿𝗶𝗻𝗴 𝗮𝗻 𝗼𝗿𝗰𝗵𝗲𝘀𝘁𝗿𝗮𝘁𝗲𝗱 𝗪𝗼𝗿𝗸𝗳𝗹𝗼𝘄 that 𝘂𝗹𝘁𝗶𝗺𝗮𝘁𝗲𝗹𝘆 𝗺𝗮𝘁𝗲𝗿𝗶𝗮𝗹𝗶𝘇𝗲𝘀 𝗮𝗻 𝗶𝗿𝗿𝗲𝘃𝗲𝗿𝘀𝗶𝗯𝗹𝗲 𝗖𝗼𝗻𝘀𝗲𝗾𝘂𝗲𝗻𝗰𝗲.</p><p>If you are not building around this chain, you are building something fragile that will eventually fail quietly — and dangerously.</p><h4>1️⃣ 𝗣𝗲𝗿𝘀𝗼𝗻𝗮 — 𝗧𝗵𝗲 𝗔𝗰𝘁𝗼𝗿 𝗧𝗵𝗮𝘁 𝗘𝘅𝗽𝗿𝗲𝘀𝘀𝗲𝘀 𝗜𝗻𝘁𝗲𝗻𝘁</h4><p>𝗘𝘃𝗲𝗿𝘆 𝘀𝘆𝘀𝘁𝗲𝗺 𝗯𝗲𝗴𝗶𝗻𝘀 𝘄𝗶𝘁𝗵 𝗮𝗻 𝗮𝗰𝘁𝗼𝗿: 𝘁𝗵𝗲 𝗣𝗲𝗿𝘀𝗼𝗻𝗮.<br> • A human user<br> • An automated agent<br> • An integrated external system<br> • A service orchestrator<br> • A machine identity</p><p>𝗧𝗵𝗲 𝗽𝗲𝗿𝘀𝗼𝗻𝗮 𝗿𝗲𝗽𝗿𝗲𝘀𝗲𝗻𝘁𝘀 𝗶𝗻𝘁𝗲𝗻𝘁 — 𝗮 𝗿𝗲𝗾𝘂𝗲𝘀𝘁 𝘁𝗼 𝗰𝗵𝗮𝗻𝗴𝗲 𝘁𝗵𝗲 𝘀𝘁𝗮𝘁𝗲 𝗼𝗳 𝘁𝗵𝗲 𝘀𝘆𝘀𝘁𝗲𝗺.<br>Core principles:</p><ul><li>Every persona must be <strong>authenticated</strong> — 𝘄𝗲 𝗸𝗻𝗼𝘄 𝘄𝗵𝗼 𝘁𝗵𝗲𝘆 𝗮𝗿𝗲.</li><li>Every persona must be <strong>authorized</strong> — 𝘄𝗲 𝗸𝗻𝗼𝘄 𝘄𝗵𝗮𝘁 𝘁𝗵𝗲𝘆’𝗿𝗲 𝗮𝗹𝗹𝗼𝘄𝗲𝗱 𝘁𝗼 𝗱𝗼.</li><li>Every persona must be <strong>audit-trailed</strong> — 𝘄𝗲 𝗸𝗻𝗼𝘄 𝘄𝗵𝗲𝗻 𝗮𝗻𝗱 𝗵𝗼𝘄 𝘁𝗵𝗲𝘆 𝗮𝗰𝘁𝗲𝗱.</li></ul><p><strong>Without explicit personas, you lose traceability, responsibility, and regulatory trust.</strong></p><h4>2️⃣ Object — The Entity Being Changed</h4><p>The <strong>persona always interacts with a specific Object — the target of change.</strong></p><ul><li>A customer profile</li><li>A payment transaction</li><li>A business order</li><li>An inventory item</li><li>A digital asset</li><li>A service request</li><li>A contract or subscription</li></ul><p><strong>The object represents the domain resource whose state your system governs.</strong></p><blockquote><strong>No object = no system. Without explicit object targeting, your system becomes chaotic, un-auditable, and impossible to debug or govern. The object is what grounds your system in real-world meaning.</strong></blockquote><h4>3️⃣ UI — The Intent Interface (Not a Logic Engine)</h4><p><strong>The User Interface</strong> — whether it’s a web form, mobile app, or external API — <strong>has one responsibility:</strong></p><ul><li><strong>Capture</strong> the <strong>persona’s intent against the object.</strong></li><li><strong>Validate</strong> that the <strong>input is complete and well-formed.</strong></li><li><strong>Submit</strong> the <strong>intent as a formal Action.</strong></li></ul><p><strong>❌ What it does not do: ❌</strong></p><ul><li>Execute business logic.</li><li>Apply state changes directly.</li><li>Bypass the chain of processing.</li></ul><blockquote><strong>The UI proposes intent. It does not execute consequences.</strong></blockquote><h4>4️⃣ Action — The Formal Request for Change</h4><p>The <strong>Action is the formalization of intent:</strong></p><ul><li><strong>Who acted?</strong></li><li><strong>Against which object?</strong></li><li><strong>What operation was requested?</strong></li><li><strong>What parameters were provided?</strong></li><li><strong>When was it submitted?</strong></li><li><strong>Under which authorization context?</strong></li></ul><p><strong>The action is immutable once submitted. It’s the sealed proposal that the system will now process.</strong> If you collapse actions directly into execution, you lose transparency, traceability, and the ability to reconstruct failures.</p><h4>5️⃣ Event — The Immutable Fact of Acceptance</h4><p>When an action is accepted, it becomes <strong>an Event — the first irreversible fact in the system:</strong></p><ul><li><strong>Timestamped</strong></li><li><strong>Immutable</strong></li><li><strong>Append-only</strong></li><li><strong>Fully auditable</strong></li><li><strong>Cryptographically</strong> verifiable (in sovereign-grade systems)</li></ul><blockquote>The <strong>event stream is the legal memory of your system</strong>. <strong>Mutable events are corruption waiting to happen</strong>. All business <strong>state reconstruction</strong>, auditing, and historical analysis depend on a clean, immutable event stream.</blockquote><h4>6️⃣ Handler — Stateless Execution Engines</h4><p><strong>Handlers are isolated components that process events.<br></strong> • <strong>Stateless</strong><br> • <strong>Idempotent</strong> (safe to retry)<br> • <strong>Deterministic</strong> (same input → same output)<br> • <strong>Fully decoupled</strong> from the UI and user requests</p><blockquote><strong>Handlers read the event and apply processing logic, preparing the system for orchestration decisions. </strong>Logic directly inside handlers should remain lightweight and focused: validation, routing, preparation — not orchestration.</blockquote><h4><strong>7️⃣ Workflow — The Business Logic Engine</strong></h4><p><strong>Real business logic lives in the Workflow layer</strong> — <strong>a state machine that knows how to:</strong></p><ul><li><strong>Sequence multi-step operations</strong></li><li><strong>Apply complex rules</strong></li><li><strong>Handle failure compensation</strong></li><li><strong>Coordinate across multiple systems or domains</strong></li><li><strong>Enforce compliance policies</strong></li><li><strong>Manage long-running processes safely</strong></li></ul><blockquote><strong>Without dedicated workflow orchestration, your system becomes brittle, opaque, and unmanageable as complexity grows. </strong>Workflows allow you to <strong>explicitly model business behavior and make it transparent, testable, and resilient.</strong></blockquote><h4>8️⃣ Consequence — The Irreversible Outcome</h4><p><strong>At the end of the chain, the workflow produces the Consequence — the irreversible update to the object’s state:<br></strong> • Balance updated<br> • Inventory reserved<br> • Order fulfilled<br> • Subscription activated<br> • Access granted<br> • Asset transferred</p><blockquote><strong>Every consequence itself becomes part of the event stream, feeding back into the system’s memory, creating a full chain of causality. No silent mutations. No blind updates. Every consequence must be traceable to its originating intent.</strong></blockquote><h4>9️⃣ Why Most Systems Fail (Quietly)</h4><p><strong>Most enterprise systems break because they ignore these boundaries:</strong></p><ol><li><strong>Inline Execution</strong>: Running business logic inside the UI or immediate API calls creates fragile coupling and invisible failures.</li><li><strong>Mutable State</strong>: Treating current state as gospel truth breaks down under corruption, concurrency, and distributed failures.</li><li><strong>Rollbacks &amp; Deletes</strong>: Deleting data or rolling back operations creates irreversible gaps in history.</li><li><strong>Blind Retries: </strong>Retrying failures without causality tracking causes duplicate transactions and state corruption.</li><li><strong>Entangled Authorization:</strong> Mixing access control directly inside business logic makes decisions opaque and un-auditable.</li></ol><blockquote><strong>These shortcuts work for prototypes. They collapse at sovereign scale.</strong></blockquote><h4><strong>🔟 What You Are Actually Building:</strong></h4><p><strong>A State Control Plane — Not Just Software</strong></p><p><strong>At true scale, enterprise architecture is not about shipping features. </strong>You are governing distributed state under risk, regulation, failure, and constant change.</p><p><strong>You are building systems that answer:</strong></p><ul><li><strong>Who acted?</strong></li><li><strong>Against what?</strong></li><li><strong>Under what rule?</strong></li><li><strong>With what outcome?</strong></li></ul><p><strong>Fully traceable, fully reproducible — forever. This is the sovereign-grade execution model.</strong></p><p>If you cannot draw <strong>a full deterministic line — from Persona → Object → Action → Event → Handler → Workflow → Consequence</strong> — <strong>then your system is architecturally incomplete.</strong></p><blockquote><strong>This is where enterprise systems mature from software into infrastructure.</strong></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b71bfe534213" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[   :     ]]></title>
            <link>https://medium.com/@eliehannouch800/-fe4a48f87f08?source=rss-79f152ec915------2</link>
            <guid isPermaLink="false">https://medium.com/p/fe4a48f87f08</guid>
            <dc:creator><![CDATA[Elie hannouch]]></dc:creator>
            <pubDate>Sun, 01 Jun 2025 10:16:16 GMT</pubDate>
            <atom:updated>2025-06-01T14:14:21.985Z</atom:updated>
            <content:encoded><![CDATA[<h3>𝗧𝗵𝗲 𝗟𝗮𝘄 𝗼𝗳 𝗦𝘆𝘀𝘁𝗲𝗺𝘀 : 𝗪𝗵𝘆 𝗘𝘅𝗰𝗲𝗹𝗹𝗲𝗻𝗰𝗲 𝗕𝗲𝗰𝗼𝗺𝗲𝘀 𝗮 𝗧𝗵𝗿𝗲𝗮𝘁</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HPEjicVCV_u0KtKfiQJicQ.png" /></figure><p>There is a brutal law that governs all complex systems — whether organizations, governments, markets, or civilizations:</p><blockquote>𝗦𝘆𝘀𝘁𝗲𝗺𝘀 𝗱𝗼 𝗻𝗼𝘁 𝗲𝘃𝗼𝗹𝘃𝗲 𝘁𝗼 𝗼𝗽𝘁𝗶𝗺𝗶𝘇𝗲 𝘁𝗿𝘂𝘁𝗵. 𝗧𝗵𝗲𝘆 𝗲𝘃𝗼𝗹𝘃𝗲 𝘁𝗼 𝗼𝗽𝘁𝗶𝗺𝗶𝘇𝗲 𝘀𝘁𝗮𝗯𝗶𝗹𝗶𝘁𝘆.</blockquote><p>This is the First Law. Every system, once formed, develops self-preservation instincts. <strong>𝗜𝘁𝘀 𝗽𝗿𝗶𝗺𝗮𝗿𝘆 𝗴𝗼𝗮𝗹 𝗯𝗲𝗰𝗼𝗺𝗲𝘀 𝗽𝗿𝗼𝘁𝗲𝗰𝘁𝗶𝗻𝗴 𝗶𝘁𝘀 𝗶𝗻𝘁𝗲𝗿𝗻𝗮𝗹 𝗲𝗾𝘂𝗶𝗹𝗶𝗯𝗿𝗶𝘂𝗺, 𝗻𝗼𝘁 𝗶𝗺𝗽𝗿𝗼𝘃𝗶𝗻𝗴 𝗶𝘁𝘀 𝗲𝘅𝘁𝗲𝗿𝗻𝗮𝗹 𝗽𝗲𝗿𝗳𝗼𝗿𝗺𝗮𝗻𝗰𝗲.</strong></p><h4>❌ <strong>𝗧𝗿𝘂𝘁𝗵 𝗱𝗲𝘀𝘁𝗮𝗯𝗶𝗹𝗶𝘇𝗲𝘀. 𝗦𝘁𝗮𝗯𝗶𝗹𝗶𝘁𝘆 𝗽𝗿𝗲𝘀𝗲𝗿𝘃𝗲𝘀</strong>. ❌</h4><blockquote><strong>𝗘𝘅𝗰𝗲𝗹𝗹𝗲𝗻𝗰𝗲, 𝗯𝘆 𝗶𝘁𝘀 𝘃𝗲𝗿𝘆 𝗻𝗮𝘁𝘂𝗿𝗲, 𝗰𝗮𝗿𝗿𝗶𝗲𝘀 𝘁𝗿𝘂𝘁𝗵 — 𝗮𝗻𝗱 𝘁𝗿𝘂𝘁𝗵, 𝗯𝘆 𝗶𝘁𝘀 𝗰𝗼𝗻𝘁𝗿𝗮𝘀𝘁, 𝗶𝘀 𝗱𝗶𝘀𝗿𝘂𝗽𝘁𝗶𝗼𝗻. 𝗡𝗼𝘁 𝗯𝘆 𝗱𝗲𝘀𝗶𝗴𝗻, 𝗯𝘂𝘁 𝗯𝘆 𝗲𝘅𝗽𝗼𝘀𝘂𝗿𝗲.</strong></blockquote><p>1. It exposes what’s broken.</p><p>2. It confronts false assumptions.</p><p>3. It refuses comfortable mediocrity.</p><p>4. It raises standards beyond what the majority can meet.</p><p>But in a system 𝘄𝗵𝗲𝗿𝗲 𝗲𝗾𝘂𝗶𝗹𝗶𝗯𝗿𝗶𝘂𝗺 𝗶𝘀 𝗽𝗿𝗶𝗼𝗿𝗶𝘁𝗶𝘇𝗲𝗱, <strong>𝘁𝗿𝘂𝘁𝗵 𝗯𝗲𝗰𝗼𝗺𝗲𝘀 𝗮 𝘀𝘆𝘀𝘁𝗲𝗺𝗶𝗰 𝘁𝗵𝗿𝗲𝗮𝘁</strong>. It creates cognitive dissonance, <strong>𝗿𝗲𝘃𝗲𝗮𝗹𝗶𝗻𝗴 𝘁𝗵𝗲 𝗴𝗮𝗽 𝗯𝗲𝘁𝘄𝗲𝗲𝗻 𝘁𝗵𝗲 𝗶𝗺𝗮𝗴𝗲 𝘁𝗵𝗲 𝘀𝘆𝘀𝘁𝗲𝗺 𝗽𝗿𝗼𝗷𝗲𝗰𝘁𝘀 — 𝗮𝗻𝗱 𝘁𝗵𝗲 𝗿𝗲𝗮𝗹𝗶𝘁𝘆 𝗶𝘁 𝗮𝗰𝘁𝘂𝗮𝗹𝗹𝘆 𝗼𝗽𝗲𝗿𝗮𝘁𝗲𝘀 𝗶𝗻.</strong></p><blockquote>❌ 𝗧𝗵𝗲 𝗴𝗿𝗲𝗮𝘁𝗲𝗿 𝘁𝗵𝗲 𝘁𝗿𝘂𝘁𝗵, 𝘁𝗵𝗲 𝗴𝗿𝗲𝗮𝘁𝗲𝗿 𝘁𝗵𝗲 𝗱𝗶𝘀𝘀𝗼𝗻𝗮𝗻𝗰𝗲. 𝗧𝗵𝗲 𝗴𝗿𝗲𝗮𝘁𝗲𝗿 𝘁𝗵𝗲 𝗱𝗶𝘀𝘀𝗼𝗻𝗮𝗻𝗰𝗲, 𝘁𝗵𝗲 𝘀𝘁𝗿𝗼𝗻𝗴𝗲𝗿 𝘁𝗵𝗲 𝗿𝗲𝘀𝗶𝘀𝘁𝗮𝗻𝗰𝗲. ❌</blockquote><p>This resistance manifests as organizational antibodies:</p><ul><li>Political maneuvering</li><li>Personal <strong>attacks masked as &quot;feedback&quot;</strong></li><li>Passive <strong>aggression framed as “collaboration”</strong></li><li>Promotions of <strong>compliant mediocrity over competent dissent</strong></li></ul><p>𝗧𝗵𝗲 𝘀𝘆𝘀𝘁𝗲𝗺 𝗽𝗿𝗲𝗳𝗲𝗿𝘀 𝗻𝗮𝗿𝗿𝗮𝘁𝗶𝘃𝗲 𝗺𝗮𝗻𝗮𝗴𝗲𝗿𝘀 𝗼𝘃𝗲𝗿 𝘁𝗿𝘂𝘁𝗵 𝘁𝗲𝗹𝗹𝗲𝗿𝘀. Because narrative managers <strong>𝗺𝗮𝗶𝗻𝘁𝗮𝗶𝗻 𝘁𝗵𝗲 𝗶𝗹𝗹𝘂𝘀𝗶𝗼𝗻 𝗼𝗳 𝗵𝗮𝗿𝗺𝗼𝗻𝘆. 𝗧𝗵𝗲𝘆 𝗸𝗲𝗲𝗽 𝘁𝗵𝗲 𝗺𝗮𝗰𝗵𝗶𝗻𝗲 𝗿𝘂𝗻𝗻𝗶𝗻𝗴 — 𝗻𝗼𝘁 𝗽𝗿𝗼𝗱𝘂𝗰𝗶𝗻𝗴, 𝗯𝘂𝘁 𝘀𝘂𝗿𝘃𝗶𝘃𝗶𝗻𝗴.</strong></p><p>Excellence, then, becomes classified as dangerous.</p><ul><li>The competent engineer <strong>is 𝗹𝗮𝗯𝗲𝗹𝗲𝗱 “𝗿𝗶𝗴𝗶𝗱.”</strong></li><li>The principled architect <strong>is 𝗹𝗮𝗯𝗲𝗹𝗲𝗱 “𝗱𝗶𝗳𝗳𝗶𝗰𝘂𝗹𝘁.”</strong></li><li>The honest operator <strong>is</strong> <strong>labeled “𝗻𝗼𝘁 𝗮 𝘁𝗲𝗮𝗺 𝗽𝗹𝗮𝘆𝗲𝗿.”</strong></li></ul><p><strong>The system does 𝗻𝗼𝘁 𝗮𝘁𝘁𝗮𝗰𝗸 𝗶𝗻𝗰𝗼𝗺𝗽𝗲𝘁𝗲𝗻𝗰𝗲 — 𝗶𝘁 𝗮𝘁𝘁𝗮𝗰𝗸𝘀 𝗲𝘅𝗽𝗼𝘀𝘂𝗿𝗲. </strong>Over time, this process becomes self-reinforcing:</p><ul><li>The best 𝗯𝘂𝗿𝗻 𝗼𝘂𝘁 𝗼𝗿 𝗹𝗲𝗮𝘃𝗲.</li><li>The mediocre rise and replicate themselves.</li><li>The organization becomes 𝗵𝗲𝗮𝘃𝗶𝗲𝗿 𝗶𝗻 𝘀𝘁𝗿𝘂𝗰𝘁𝘂𝗿𝗲, 𝗹𝗶𝗴𝗵𝘁𝗲𝗿 𝗶𝗻 𝘀𝘂𝗯𝘀𝘁𝗮𝗻𝗰𝗲.</li><li>Noise replaces signal. Form replaces function.</li></ul><p>And then one day, when external reality finally collides with internal decay — when competition, innovation, or market collapse arrives — the system faces a reckoning it can no longer defer. 𝗕𝘆 𝘁𝗵𝗲𝗻, 𝘁𝗵𝗲 𝗽𝗲𝗼𝗽𝗹𝗲 𝘄𝗵𝗼 𝗰𝗼𝘂𝗹𝗱 𝗵𝗮𝘃𝗲 𝘀𝗮𝘃𝗲𝗱 𝗶𝘁 𝗮𝗿𝗲 𝗹𝗼𝗻𝗴 𝗴𝗼𝗻𝗲.</p><blockquote>The tragedy isn’t that the system failed. The tragedy is 𝘁𝗵𝗮𝘁 𝘁𝗵𝗲 𝘀𝘆𝘀𝘁𝗲𝗺 𝗰𝗵𝗼𝘀𝗲 𝗳𝗮𝗶𝗹𝘂𝗿𝗲 𝗯𝘆 𝗽𝗿𝗼𝘁𝗲𝗰𝘁𝗶𝗻𝗴 𝗶𝘁𝘀 𝗰𝗼𝗺𝗳𝗼𝗿𝘁 𝗼𝘃𝗲𝗿 𝗶𝘁𝘀 𝗰𝗼𝗺𝗽𝗲𝘁𝗲𝗻𝗰𝗲. ‼️</blockquote><p>This is not an organizational problem. 𝗧𝗵𝗶𝘀 𝗶𝘀 𝗮 𝗹𝗮𝘄 𝗼𝗳 𝘀𝘆𝘀𝘁𝗲𝗺𝘀 — 𝗼𝗯𝘀𝗲𝗿𝘃𝗮𝗯𝗹𝗲 𝗮𝗰𝗿𝗼𝘀𝘀 𝗵𝗶𝘀𝘁𝗼𝗿𝘆, 𝗲𝗺𝗽𝗶𝗿𝗲𝘀, 𝗺𝗮𝗿𝗸𝗲𝘁𝘀, 𝗮𝗻𝗱 𝗰𝗼𝗿𝗽𝗼𝗿𝗮𝘁𝗶𝗼𝗻𝘀 𝗮𝗹𝗶𝗸𝗲. The few organizations that break this law?</p><p>They are not the norm — 𝘁𝗵𝗲𝘆 𝗮𝗿𝗲 𝘁𝗵𝗲 𝗲𝘅𝗰𝗲𝗽𝘁𝗶𝗼𝗻. <strong>They are led by the rare few 𝘄𝗵𝗼 𝘂𝗻𝗱𝗲𝗿𝘀𝘁𝗮𝗻𝗱 𝘁𝗵𝗮𝘁 𝗽𝗿𝗼𝘁𝗲𝗰𝘁𝗶𝗻𝗴 𝘁𝗿𝘂𝘁𝗵 — 𝗵𝗼𝘄𝗲𝘃𝗲𝗿 𝘂𝗻𝗰𝗼𝗺𝗳𝗼𝗿𝘁𝗮𝗯𝗹𝗲 — 𝗶𝘀 𝘁𝗵𝗲 𝗼𝗻𝗹𝘆 𝗿𝗲𝗮𝗹 𝗱𝗲𝗳𝗲𝗻𝘀𝗲 𝗮𝗴𝗮𝗶𝗻𝘀𝘁 𝗰𝗼𝗹𝗹𝗮𝗽𝘀𝗲.</strong></p><p>Elie Hannouch, 01/06/2025.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fe4a48f87f08" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Build for When, Not Just What: Temporal-First System Design]]></title>
            <link>https://medium.com/@eliehannouch800/build-for-when-not-just-what-temporal-first-system-design-0192bc0e7349?source=rss-79f152ec915------2</link>
            <guid isPermaLink="false">https://medium.com/p/0192bc0e7349</guid>
            <category><![CDATA[governance]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[mongodb]]></category>
            <dc:creator><![CDATA[Elie hannouch]]></dc:creator>
            <pubDate>Thu, 29 May 2025 19:39:19 GMT</pubDate>
            <atom:updated>2025-05-29T20:47:47.831Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5-vpAXio9C-rnnLivxuLaw.png" /></figure><p>Most systems were built for “what happened” — not <em>when</em>, <em>why</em>, or <em>under what conditions</em>. They treat time as an afterthought: a field in a schema, not a first-class actor in the architecture. But if your system can’t reconstruct how it reached a decision — or verify the context an AI used to act — then it’s not just incomplete. It’s untrustworthy.</p><p>In this piece, we challenge the foundations of CRUD-based thinking and introduce a temporal-first design paradigm. MongoDB becomes more than a document store — it evolves into a deterministic memory plane. One where data is immutable, snapshots are composable, and causality is explicit. This is the architecture for systems that must <strong>prove</strong> what they knew — and <strong>when</strong> they knew it.</p><h3>I. Premise: Time is Not a Dimension — It’s a Dependency Graph</h3><p>Traditional systems treat time as a secondary attribute — a timestamp on a row, a column called lastModified, a soft field in the background of a table schema. But in dynamic, high-integrity, distributed systems, <strong>time is not a field; it is the backbone of data coherence</strong>.</p><p>In mutable document stores, state is a lossy snapshot. When systems rely on live state without temporal context, they lose the ability to reason about causality, concurrency, or regulatory intent.</p><p>Temporal modeling in MongoDB isn’t about adding a timestamp. It’s about:</p><ul><li>Storing <strong>chains of immutable state</strong>.</li><li>Resolving <strong>snapshot subgraphs</strong>.</li><li>Binding <strong>external actions (agents, workflows, rules)</strong> to a <strong>versioned worldview</strong>.</li></ul><p>This means rearchitecting MongoDB from a persistence layer to a <strong>deterministic memory engine</strong>.</p><h3>II. Temporal Memory Plane Design: Core Concepts</h3><h4>1. Immutable Version Lineage</h4><p>At the heart of temporal modeling lies a radical shift: <strong>state is no longer overwritten — it is evolved.</strong> Every change to an entity becomes a new, immutable version, creating a <strong>chain of truth</strong> rather than a series of destructive updates. This transforms your database from a mutable store into a <strong>historical record of intent</strong>.</p><p>Key principles:</p><ul><li><strong>No Overwrites</strong>: Once written, a version is immutable — forever. There is no “update,” only the creation of a new version with an explicit parent.</li><li><strong>Composite Identity</strong>: Each version is uniquely identified by a { entityId, version } pair, ensuring clarity in lineage and consistency in retrieval.</li><li><strong>Append-Only Log Structure</strong>: The system appends new versions to the end of the entity’s history — no destructive operations, no partial state loss, and no ambiguity.</li></ul><p>This structure creates a <strong>version graph</strong> — specifically a <strong>Directed Acyclic Graph (DAG)</strong> — where:</p><ul><li>Every node is a specific version of an entity.</li><li>Every edge represents a mutation, a transition from one version to the next.</li><li>The graph is <strong>forward-only</strong>, ensuring that time always progresses and causality is preserved.</li><li>The graph can be <strong>traversed</strong> to inspect history, <strong>hashed</strong> to verify integrity, and <strong>validated</strong> to enforce structural constraints (e.g., no cycles, missing links, or orphan versions).</li></ul><blockquote><em>Think of this as Git for your data — except with cryptographic consistency, audit-grade traceability, and real-world semantics embedded in every change.</em></blockquote><h4>2. Snapshot Subgraph Resolution</h4><p>In high-integrity systems, actions don’t occur in isolation. Decisions — whether by a human, a microservice, or an AI model — are made in the context of <strong>multiple entities, each at a specific point in time</strong>. That context must be explicit, sealed, and replayable. This is where <strong>snapshot subgraph resolution</strong> becomes foundational.</p><p>A <strong>snapshot</strong> is not a single record — it is a <strong>composite, version-bound subgraph</strong> of all the entities involved in a particular decision, transaction, or event. It captures the <strong>exact worldview</strong> that was known and available at the moment of execution. Each of these entities exists at a specific version. A snapshot binds them:</p><pre>{<br>  &quot;snapshotId&quot;: &quot;snp_983dd&quot;,<br>  &quot;graph&quot;: [<br>    { &quot;entity&quot;: &quot;case&quot;, &quot;id&quot;: &quot;case_112&quot;, &quot;v&quot;: 6 },<br>    { &quot;entity&quot;: &quot;user&quot;, &quot;id&quot;: &quot;usr_92&quot;, &quot;v&quot;: 2 },<br>    { &quot;entity&quot;: &quot;file&quot;, &quot;id&quot;: &quot;doc_204&quot;, &quot;v&quot;: 9 }<br>  ]<br>}</pre><p>This snapshot binds together:</p><ul><li>A <strong>case</strong> in version 6 (perhaps after a series of hearings),</li><li>A <strong>user</strong> in version 2 (who may have just been assigned new roles),</li><li>A <strong>document</strong> in version 9 (recently revised or notarized).</li></ul><p>Each of these versions may have their own prior histories — but the snapshot pins them to a <strong>frozen, deterministic view</strong>.</p><h4>3. Agentic Execution Anchoring</h4><p>In temporal-first architectures, autonomous agents (LLMs, rule engines, microservices) must execute deterministically within a <strong>bounded, version-controlled context</strong>. Agentic execution anchoring ensures that agents operate strictly on <strong>sealed snapshots</strong> — resolved subgraphs of entities with exact version bindings — to prevent data drift, non-determinism, and silent divergence from ground truth.</p><h4>🔒 Sealed Context Guarantee</h4><p>Each agent execution is scoped to a resolved snapshot:</p><pre>{<br>  &quot;snapshotId&quot;: &quot;snap_983dd&quot;,<br>  &quot;graph&quot;: [<br>    { &quot;entity&quot;: &quot;case&quot;, &quot;id&quot;: &quot;case_112&quot;, &quot;v&quot;: 6 },<br>    { &quot;entity&quot;: &quot;user&quot;, &quot;id&quot;: &quot;usr_92&quot;, &quot;v&quot;: 2 },<br>    { &quot;entity&quot;: &quot;doc&quot;, &quot;id&quot;: &quot;doc_204&quot;, &quot;v&quot;: 9 }<br>  ]<br>}</pre><p>This snapshot is <strong>sealed</strong> before agent invocation, typically via SHA-256 hashing of the canonical JSON form. Optionally, the snapshot can be digitally signed. No agent has read access outside this sealed subgraph.</p><h4>📄 Execution Contract Logging</h4><p>Each agent action is logged with full provenance and hash validation:</p><pre>{<br>  &quot;agentContextId&quot;: &quot;actx_42394&quot;,<br>  &quot;snapshotId&quot;: &quot;snap_983dd&quot;,<br>  &quot;inputHash&quot;: &quot;sha256:abc...&quot;,<br>  &quot;output&quot;: {<br>    &quot;decision&quot;: &quot;approve&quot;,<br>    &quot;confidence&quot;: 0.92<br>  },<br>  &quot;outputHash&quot;: &quot;sha256:def...&quot;,<br>  &quot;executedBy&quot;: &quot;agent_claude_v3&quot;,<br>  &quot;executedAt&quot;: &quot;2025-05-29T09:14:00Z&quot;<br>}</pre><p>This allows for:</p><ul><li><strong>Full reproducibility</strong>: Re-executing the agent with the same snapshot must yield the same output.</li><li><strong>Diffing</strong>: Any deviation in output under the same input is detectable.</li><li><strong>Access control enforcement</strong>: Agents cannot issue queries outside the sealed snapshot; violations are rejected at runtime.</li><li><strong>Deterministic Execution Path</strong>: No external DB reads, no mutable access — only sealed data as input.</li><li><strong>Causal Isolation</strong>: The snapshotId becomes the root of trust for all downstream effects, ensuring that decisions are reproducibly bound to specific historical states.</li><li><strong>Observability Hooks</strong>: Each execution emits telemetry (AgentExecuted, SnapshotResolved, etc.) to observability platforms (Prometheus, OpenTelemetry).</li><li><strong>Tamper Detection</strong>: Divergence between inputHash and reconstructed input snapshot flags unauthorized mutations or inconsistencies.</li></ul><h3>III. MongoDB as a Temporal Execution Engine</h3><p>MongoDB, while often used as a general-purpose document store, provides a rich set of primitives that — when composed intentionally — can form the backbone of a <strong>high-performance, causally consistent, temporally-aware execution system</strong>. To realize this, one must stop treating MongoDB as a simple CRUD database and start leveraging it as a <strong>deterministic, append-only memory engine</strong>.</p><p><strong>Core Temporal Primitives and Their Roles</strong></p><h4><strong>1. $graphLookup</strong></h4><ul><li>Enables recursive traversal of version dependencies across entities.</li><li>Used to resolve the full version lineage or dependency closure of a snapshot graph.</li><li>Critical for computing transitive dependencies (e.g., case → documents → users).</li><li>Supports deep snapshot validation and closure enforcement.</li></ul><pre>db.entities.aggregate([<br>  {<br>    $graphLookup: {<br>      from: &quot;entityVersions&quot;,<br>      startWith: &quot;$versionRefs&quot;,<br>      connectFromField: &quot;versionRefs&quot;,<br>      connectToField: &quot;entityId&quot;,<br>      as: &quot;resolvedGraph&quot;<br>    }<br>  }<br>])</pre><h4>2. <strong>$merge</strong></h4><ul><li>Materializes views (current or frozen) into target collections.</li><li>Enables <strong>snapshot freezing</strong> or <strong>state promotion</strong> from versioned documents.</li><li>Can be used to create a point-in-time mirror for downstream agents or UIs.</li></ul><pre>db.versionedSnapshots.aggregate([<br>  { $match: { snapshotId: &quot;snap_4491a2&quot; } },<br>  { $merge: { into: &quot;frozenSnapshots&quot;, whenMatched: &quot;replace&quot; } }<br>])</pre><h4><strong>3. $unionWith</strong></h4><ul><li>Supports timeline comparisons and historical joins across different version states.</li><li>Used for <strong>diff views</strong>, <strong>audit timelines</strong>, or <strong>change analysis</strong> across versions.</li><li>Allows querying multiple snapshots in a single pipeline.</li></ul><pre>db.entityVersions.aggregate([<br>  { $match: { entityId: &quot;user_239&quot;, version: { $in: [2, 3] } } },<br>  { $unionWith: { coll: &quot;entityVersions&quot;, pipeline: [{ $match: { version: 4 } }] } }<br>])</pre><h4>4. <strong>Change Streams</strong></h4><ul><li>Observes deltas in real time for syncing, audit trails, and reactive processing.</li><li>Enables downstream systems to listen for new mutations, sealed snapshots, or agent actions.</li><li>Used for real-time propagation to cache layers, analytics systems, or event buses.</li></ul><pre>db.entityVersions.watch([<br>  { $match: { &quot;operationType&quot;: &quot;insert&quot; } }<br>])</pre><h4>5. <strong>Queryable Encryption (FLE2)</strong></h4><ul><li>Enforces fine-grained access control and compliance via field-level encryption.</li><li>Supports <strong>scoped visibility</strong> of sensitive fields based on role or context.</li><li>Ensures that even if a snapshot is accessible, individual fields may be protected or redacted.</li></ul><pre>{<br>  &quot;email&quot;: {<br>    &quot;encryptedValue&quot;: &quot;AEAD_AES_256_CBC_HMAC_SHA_512-Random&quot;,<br>    &quot;visibleTo&quot;: [&quot;supervisor&quot;, &quot;auditor&quot;]<br>  }<br>}</pre><h3>IV. Mutation Layer Design</h3><p>In a temporal-first system, <strong>mutations are not just data changes</strong> — they are explicit declarations of <strong>intent</strong>, <strong>cause</strong>, and <strong>version lineage</strong>. Every change becomes a formal, typed contract that captures <em>what</em> changed, <em>why</em> it changed, <em>from which exact version</em>, and <em>under whose authority</em>. This isn’t a diff — it&#39;s a <strong>verifiable narrative of evolution</strong>.</p><p>Each mutation event must include:</p><ul><li>mutationId: A globally unique identifier for traceability.</li><li>entityId: The identity of the entity being mutated.</li><li>fromVersion and toVersion: Ensures optimistic concurrency and blocks silent overwrites. This establishes strict lineage and protects against race conditions.</li><li>type: A domain-specific label (e.g., term_amendment, status_transition) that adds <strong>semantic meaning</strong> to the change.</li><li>delta: A structured, schema-compliant diff ($set, $unset, etc.), which can be replayed, validated, or reversed.</li><li>triggeredBy: Links the mutation to an external workflow, user action, or automated rule — grounding the mutation in a causality chain.</li></ul><pre>{<br>  &quot;mutationId&quot;: &quot;mut_9821&quot;,<br>  &quot;entityId&quot;: &quot;contract_292&quot;,<br>  &quot;fromVersion&quot;: 6,<br>  &quot;toVersion&quot;: 7,<br>  &quot;type&quot;: &quot;term_amendment&quot;,<br>  &quot;delta&quot;: {<br>    &quot;$set&quot;: { &quot;interestRate&quot;: &quot;4.2%&quot; }<br>  },<br>  &quot;triggeredBy&quot;: &quot;workflow_v21&quot;<br>}</pre><p>Mutations are used to:</p><ul><li>Derive new versions</li><li>Materialize diffs</li><li>Construct decision causality chains</li></ul><h3>V. Isolation and Causality at Architectural Level</h3><p>To protect against concurrency errors, race conditions, or silent overwrites:</p><h4>Write-Forward with Optimistic Locks</h4><ul><li>Writes must declare fromVersion</li><li>If currentVersion != fromVersion, write is rejected</li></ul><h4>Composable Snapshots</h4><ul><li>Snapshots are recursive and closed</li><li>Sealed snapshots are hash-consistent and optionally signed</li><li>Can be resolved via $graphLookup</li></ul><h4>Causal Flow Anchoring</h4><ul><li>Downstream systems receive the snapshotId</li><li>Actions are bound to <strong>exact input context</strong></li></ul><h3>VI. Query Model Rewrites</h3><p>Querying live mutable state is <strong>explicitly disallowed</strong> for critical decisions, workflows, or agents. Instead, all reads are rewritten to resolve against <strong>sealed version graphs</strong>, ensuring determinism, reproducibility, and traceable execution.</p><p>This means shifting from querying <strong>“what is”</strong> to querying <strong>“what was known at the time of action”</strong> — a foundational shift for causality-bound systems.</p><p><strong>Instead of:</strong></p><pre>db.users.find({ _id: &quot;usr_92&quot; }) // ⚠️ Current state — mutable, unsafe</pre><p><strong>You perform:</strong></p><pre>resolveGraph(snapshotId).then(graph =&gt; {<br>  fetchEntities(graph) // Pull versioned documents from entityVersionStore<br>})</pre><p><strong>Each entity in the graph carries an exact version:</strong></p><pre>[<br>  { &quot;entity&quot;: &quot;user&quot;, &quot;id&quot;: &quot;usr_92&quot;, &quot;version&quot;: 2 },<br>  { &quot;entity&quot;: &quot;case&quot;, &quot;id&quot;: &quot;case_112&quot;, &quot;version&quot;: 6 }<br>]</pre><p>This ensures <strong>zero ambiguity</strong> in data access — every field, every value, and every relationship is resolved from a <strong>sealed, immutable snapshot.</strong></p><h3>VII. Observability: Architected Trace Surfaces</h3><p>Observability is not just about monitoring performance or uptime — it’s about <strong>reconstructing causality</strong>, <strong>auditing decisions</strong>, and <strong>tracing how every version, mutation, and agent action unfolded across time</strong>. To support this, observability must be <strong>architected as a first-class data plane</strong>, emitting structured, timestamped, and semantically rich traces at every critical point in the execution lifecycle.</p><h4>📡 Event Emission Points</h4><p>Each core operation in the system emits a trace event with unique identifiers, timestamps, hash validations, and provenance metadata:</p><ul><li><strong>VersionCreated</strong> is emitted each time a new immutable version of an entity is written to the system. This marks the successful persistence of a state transition and its attachment to the version lineage.</li><li><strong>MutationContractLogged</strong> signals that a typed mutation event has been generated, schema-validated, and formally recorded. It confirms the mutation&#39;s integrity and role in deriving a new version.</li><li><strong>SnapshotResolved</strong> is emitted when a snapshot subgraph — a set of interrelated entity versions — is fully resolved, sealed, and optionally hashed. This guarantees that the snapshot is consistent and self-contained.</li><li><strong>AgentExecuted</strong> occurs when an autonomous agent (like an LLM, rules engine, or microservice) completes an action based on a sealed snapshot. It includes references to the snapshot used, input/output hashes, and execution metadata.</li><li><strong>SnapshotDiffed</strong> is emitted when two snapshots are structurally or semantically compared, producing a diff that can be used for audit, debugging, or impact analysis of data evolution.</li></ul><pre>{<br>  &quot;eventType&quot;: &quot;AgentExecuted&quot;,<br>  &quot;agentContextId&quot;: &quot;actx_39218&quot;,<br>  &quot;snapshotId&quot;: &quot;snap_4491a2&quot;,<br>  &quot;inputHash&quot;: &quot;sha256-abc...&quot;,<br>  &quot;outputHash&quot;: &quot;sha256-def...&quot;,<br>  &quot;executedBy&quot;: &quot;agent_claude_v3&quot;,<br>  &quot;timestamp&quot;: &quot;2025-05-29T12:01:00Z&quot;,<br>  &quot;durationMs&quot;: 58<br>}</pre><h4>Observability as System Memory</h4><p>This architecture transforms observability from a side-channel into a <strong>formal, queryable narrative</strong> of system behavior:</p><ul><li>You can <strong>diff two decisions</strong> and see <em>exactly what changed</em> in the input.</li><li>You can <strong>trace causality</strong> from a mutation to every downstream action it triggered.</li><li>You can <strong>replay agent executions</strong> for audit or debugging, down to the byte-level input hash.</li></ul><p><strong>Stream To</strong></p><p>To make these trace events actionable, they are <strong>streamed in real time</strong> to both system-level and intelligence platforms:</p><ul><li><strong>Prometheus / Grafana: </strong>Track latency, mutation throughput, agent execution durations, snapshot resolution trends.</li><li><strong>OpenTelemetry: </strong>Trace individual operations across services for distributed system debugging, agent input lineage, and mutation graph propagation.</li><li><strong>Audit Logs: </strong>All critical operations are immutably logged for forensic review, legal compliance, or third-party audit.</li><li><strong>Vector Stores for LLM Traceability: </strong>Agent input/output pairs, along with snapshot contexts, are indexed in vector databases.</li></ul><h3>VIII. What You Must Build</h3><h4>Versioned Entity Store</h4><p><strong>Purpose:</strong> Stores immutable, append-only records for every entity version. Guarantees traceability and historical fidelity. Supports hash verification and can be used to reconstruct any past system state deterministically.</p><pre>{<br>  &quot;entityId&quot;: &quot;user_239&quot;,<br>  &quot;version&quot;: 3,<br>  &quot;data&quot;: {<br>    &quot;name&quot;: &quot;Jane Doe&quot;,<br>    &quot;email&quot;: &quot;jane.doe@example.com&quot;<br>  },<br>  &quot;createdAt&quot;: ISODate(&quot;2025-05-29T08:00:00Z&quot;),<br>  &quot;hash&quot;: &quot;sha256-abc...&quot;<br>}</pre><p><strong>Design:</strong></p><ul><li>Each entity type (e.g., user, case, document) has its own versioned collection.</li><li>Each version is uniquely identified by a compound key: { entityId, version }.</li><li>A canonical hash is calculated per version using SHA-256 on a canonical JSON serialization.</li><li>Every version document includes createdAt, createdBy, reason, and optional references to mutation contracts.</li></ul><p><strong>Responsibilities:</strong></p><ul><li>Ensure temporal consistency and traceability.</li><li>Support efficient lookups of latest and historical versions.</li><li>Maintain cryptographic integrity for legal audit purposes</li></ul><h4>Snapshot Resolver</h4><p><strong>Purpose:</strong> Dynamically assembles multi-entity, version-specific subgraphs to support decision snapshots, AI inference contexts, and workflow consistency. Optimized to detect cycles, stale references, and to validate dependency closure.</p><pre>{<br>  &quot;snapshotId&quot;: &quot;snap_4491a2&quot;,<br>  &quot;graph&quot;: [<br>    { &quot;entity&quot;: &quot;case&quot;, &quot;id&quot;: &quot;case_124&quot;, &quot;version&quot;: 5 },<br>    { &quot;entity&quot;: &quot;document&quot;, &quot;id&quot;: &quot;doc_76&quot;, &quot;version&quot;: 2 }<br>  ],<br>  &quot;sealedAt&quot;: ISODate(&quot;2025-05-29T08:12:00Z&quot;)<br>}</pre><p><strong>Design:</strong></p><ul><li>Resolves entity graphs recursively using $graphLookup or foreign key mappings.</li><li>Detects and resolves version dependencies, ensuring snapshot closure.</li><li>Can optionally freeze the graph with a snapshotHash and digital signature.</li></ul><p><strong>Responsibilities:</strong></p><ul><li>Guarantee snapshot coherence across complex domain models.</li><li>Validate snapshot completeness and dependency graphs.</li><li>Serve as a base for AI agents, legal decisions, and workflow triggers.</li></ul><h4>Mutation Compiler</h4><p><strong>Purpose:</strong> Converts domain events into low-level mutations. Handles schema validation, patch generation, transformation into diff contracts, and prepares change logs for audit and replay engines.</p><pre>{<br>  &quot;event&quot;: &quot;contract_approved&quot;,<br>  &quot;mutation&quot;: {<br>    &quot;$set&quot;: { &quot;status&quot;: &quot;approved&quot;, &quot;approvedBy&quot;: &quot;user_882&quot; }<br>  },<br>  &quot;target&quot;: &quot;contract_392&quot;,<br>  &quot;fromVersion&quot;: 2,<br>  &quot;toVersion&quot;: 3<br>}</pre><p><strong>Design:</strong></p><ul><li>Receives a business-level event (e.g., “approve contract”).</li><li>Fetches the last known version.</li><li>Produces a $set, $unset, or custom delta diff.</li><li>Links to fromVersion, toVersion, and embeds intent metadata.</li></ul><p><strong>Responsibilities:</strong></p><ul><li>Serve as the single source of truth for how state transitions occur.</li><li>Validate structural and domain rules before generating the next version.</li><li>Write metadata-rich mutation contracts to a dedicated collection.</li></ul><h4>Agent Context Engine</h4><p><strong>Purpose:</strong> Logs and enforces the constraints of AI and system agents by binding them to explicit snapshots. Ensures reproducibility of their actions and validates that decisions are always made from sealed contexts.</p><pre>{<br>  &quot;agentId&quot;: &quot;claude_v3&quot;,<br>  &quot;snapshotId&quot;: &quot;snap_4491a2&quot;,<br>  &quot;inputHash&quot;: &quot;sha256-xyz...&quot;,<br>  &quot;action&quot;: &quot;recommendApproval&quot;,<br>  &quot;output&quot;: { &quot;decision&quot;: &quot;approve&quot;, &quot;confidence&quot;: 0.92 }<br>}</pre><p><strong>Design:</strong></p><ul><li>Associates every action with a sealed snapshot.</li><li>Captures the LLM input hash and output.</li><li>Enforces read-only access to the snapshot and rejects out-of-scope queries.</li></ul><p><strong>Responsibilities:</strong></p><ul><li>Prevent hallucinations and drift by locking agent access to temporal truth.</li><li>Log all actions for accountability, replay, or debugging.</li><li>Enable regulatory introspection and AI safety audits.</li></ul><h4>Temporal Query Layer</h4><p><strong>Purpose:</strong> Provides composable APIs for querying versioned documents, comparing snapshots, reconstructing time-travel views, and exposing semantic diffs between versions. Supports read isolation and historical joins.</p><pre>findVersionedEntity(&quot;user_239&quot;, { asOf: ISODate(&quot;2025-05-29T08:00:00Z&quot;) })</pre><p><strong>Design:</strong></p><ul><li>Exposes APIs like getVersionedEntity(id, version), getSnapshot(snapshotId).</li><li>Wraps $unionWith and $facet to construct diffs and explain data evolution.</li></ul><p><strong>Responsibilities:</strong></p><ul><li>Resolve historical and real-time query use cases.</li><li>Enable UI rendering of version differences and lifecycle timelines.</li><li>Power AI grounding with schema-enforced queries.</li></ul><h4>Governance Overlay</h4><p><strong>Purpose: </strong>Enforces access rules, field-level encryption policies (e.g., via FLE2), redaction filters, and dynamic role-based access control on versioned documents and snapshot queries. Supports zero-trust environments.</p><pre>A supervisor can see email, but a viewer gets redacted: { &quot;email&quot;: &quot;****&quot; }</pre><p><strong>Design:</strong></p><ul><li>Integrates with MongoDB FLE2 for field-level encryption.</li><li>Redacts sensitive fields dynamically based on RBAC and scopes.</li><li>Logs access for audit trails.</li></ul><p><strong>Responsibilities:</strong></p><ul><li>Protect regulated data (PII, financial records) under compliance mandates.</li><li>Differentiate access rules across entity roles and lifecycle phases.</li><li>Apply field masks or tokenization as needed.</li></ul><h4>Causal Event Router</h4><p><strong>Purpose:</strong> Orchestrates downstream message routing and workflow propagation based on causally linked snapshot decisions. Anchors notifications, side effects, and external triggers to versioned input contexts.</p><pre>{<br>  &quot;workflow&quot;: &quot;initiate_settlement&quot;,<br>  &quot;triggeredBy&quot;: &quot;approval_event&quot;,<br>  &quot;snapshotId&quot;: &quot;snap_4491a2&quot;,<br>  &quot;entityRef&quot;: { &quot;id&quot;: &quot;contract_391&quot;, &quot;version&quot;: 4 }<br>}</pre><p><strong>Design:</strong></p><ul><li>Listens to mutations.</li><li>Emits event payloads tagged with origin snapshotId.</li><li>Supports parallel downstream processing with full context.</li></ul><p><strong>Responsibilities:</strong></p><ul><li>Maintain cause-effect lineage for workflow propagation.</li><li>Allow external systems to replay or retry based on version source.</li><li>Enable external orchestrators to subscribe to state graph events.</li></ul><h3>Final Principle: State Without Time is Meaningless</h3><p>If your system forgets what it knew, when it knew it, and why it acted — it is not trustworthy. MongoDB has every primitive needed to power coherent, secure, agent-aligned systems. But only if you stop building for “now” — and start building for <strong>truth under time.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0192bc0e7349" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Inside the Engine: The Sub-Millisecond Performance Relay of MongoDB 8.0]]></title>
            <link>https://medium.com/@eliehannouch800/inside-the-engine-the-sub-millisecond-performance-relay-of-mongodb-8-0-267d40d358f5?source=rss-79f152ec915------2</link>
            <guid isPermaLink="false">https://medium.com/p/267d40d358f5</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[mongodb]]></category>
            <category><![CDATA[nosql]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[persformance]]></category>
            <dc:creator><![CDATA[Elie hannouch]]></dc:creator>
            <pubDate>Tue, 20 May 2025 11:15:24 GMT</pubDate>
            <atom:updated>2025-10-17T15:21:32.742Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rM-JIRWo49QIKv-vZ4taEQ.png" /></figure><p>In environments where microseconds dictate competitive advantage, MongoDB 8.0 delivers a meticulously tuned execution pipeline that transforms raw network packets into sub-millisecond query responses at global scale. This reference traces a single trade query through every internal boundary network ingress, scheduling, security, parsing, planning, execution, storage‐engine internals, indexing, replication, sharding, change streams, time‐series buckets, backup, and monitoring illustrating how MongoDB 8.0’s per-CPU allocators, active-work profiling, SIMD-vectorized execution, adaptive bucketization, compact resume tokens, and refined journaling coalesce into a seamless, predictable performance engine.</p><h4>Stage 1: Network Arrival &amp; Task Dispatch</h4><p>At <strong>09:30:45.123 UTC</strong>, your Node.js driver pulls a TLS session from its pool and emits:</p><pre>db.trades.find({ symbol: &quot;AAPL&quot; })<br>         .sort({ timestamp: -1 })<br>         .limit(10);</pre><p>The NIC DMA’s the encrypted packet into kernel memory and, within microseconds, MongoDB’s ASIO reactor (mongo::transport::ServiceEntryPoint) zero-copies it into a pre-allocated SocketFrame. That frame lands on the TaskExecutor’s lock-free queue, waking a parked worker thread in under 10 µs. With network I/O complete, control transfers seamlessly to scheduling.</p><h4>Stage 2: OperationContext &amp; ACL/Parsing</h4><p>The awakened thread immediately instantiates an OperationContext containing your session’s causal clusterTime, read/write concerns, transaction state, and kill-operation tokens. Reusing the TLS connection lets the AuthorizationManager return an ACL verdict in ~200 µs from its SCRAM cache. With permissions verified, the raw BSON enters Command::parse(), unfolding into an AST. JSON Schema validators fire against any collection rules, and the AST canonicalizes normalizing filters, pushing down projections, extracting sort keys before computing a 64-bit fingerprint for the PlanCache. Having canonicalized the query, we now pass the baton to query planning.</p><h4>Stage 3: PlanCache Lookup &amp; Query Planning</h4><p>That fingerprint is looked up in PlanCacheImpl. On a cache hit, the cached SBE plan rehydrates instantly and bypasses planning. On a miss, QueryPlanner generates candidates a full-collection scan, an index scan on { symbol: 1 }, and a compound index scan on { symbol: 1, timestamp: -1 }. It trial-runs each against the first 128 documents, capturing keysExamined and docsReturned. The fastest contender compiles into SIMD-vectorized SBE bytecode with in-lined numeric filters and any $convert/$toUUID operators annotated for runtime. Armed with an optimized plan, execution now commences.</p><h4>Stage 4: SBE Execution &amp; Cooperative Yielding</h4><p>The SBE engine executes the compiled bytecode, traversing WiredTiger B-tree pages. Every 128 documents or when lock-wait thresholds trigger it cooperatively yields (via internalQueryExecYieldIterations), granting CPU slices to concurrent writes. Upon completion, CurOp::complete() aggregates <strong>active-work latency</strong> (excluding lock-wait and journal delays in 8.0), CPU time, I/O counts, and returned-document metrics. If active-work latency exceeds a 2 ms SLA or matches a 10 % sample rate, an atomic profiling document is written to system.profile, maintaining precise diagnostics. With execution metrics captured, the baton transfers to the storage engine.</p><h4>Stage 5: WiredTiger MVCC, Cache &amp; Journaling</h4><p>Under the surface, WiredTiger’s MVCC gives each operation its own read snapshot, so readers never block writers; old document versions stream into the <strong>history store</strong> until eviction threads merge them back into pages. The WT cache sized to 60 % of RAM monitors dirty pages at an 8 % threshold, flushing asynchronously to avoid foreground stalls. Writes append to the journal file, and a timer thread fsyncs every 20 ms (commitIntervalMs), bounding durability latency; 8.0’s active-work profiling ensures these fsync waits do not appear in slow-op logs. Meanwhile, per-CPU TCMalloc caches minimize fragmentation on your multi-socket servers. Having persisted and profiled the operation, we transition to indexing.</p><h4>Stage 6: Index Mastery &amp; Pre-Splits</h4><p>Your compound index { symbol: 1, timestamp: -1, price: 1 } adheres to the ESR (Equality → Sort → Range) rule, allowing the SBE engine to satisfy the query with a single index scan. You pre-split hot key ranges—invoking:</p><pre>sh.splitAt(&quot;market.trades&quot;, { symbol: &quot;H&quot; });<br>sh.splitAt(&quot;market.trades&quot;, { symbol: &quot;M&quot; });<br>// …and so forth through &quot;Z&quot;</pre><p>eliminating runtime page splits. A VIP partial index on high-price trades:</p><pre>db.trades.createIndex(<br>  { price: -1 },<br>  { partialFilterExpression: { price: { $gt: 1000 } } }<br>);</pre><p>ensures premium fetches hit a covered-index probe, bypassing the document layer entirely. With index probes optimized, replication and transactional guarantees take over.</p><h4>Stage 7: Replication &amp; Transactional Guarantees</h4><p>Upon commit, the primary writes the corresponding oplog entry to local.oplog.rs and streams it to secondaries within 10 ms. Under a w: &quot;majority&quot; write concern, the majority-commit point advances only after a quorum acknowledges, safeguarding against partitions. If executed within a multi-document transaction, MongoDB’s two-phase commit protocol prepares changes on each shard’s journal and issues the global commit minimizing cross-shard latency by keeping the prepare window razor-thin. Next, mongos routing and shard-aware dispatch refine the query’s scope.</p><h4>Stage 8: Mongos Routing &amp; Sharding Precision</h4><p>Your mongos routers, armed with a 30 s-TTL CatalogCache of config.collections and config.chunks, resolve your hybrid shard key { region: 1, tradeId: &quot;hashed&quot; } plus GDPR-compliant tag ranges (EU vs. APAC). Queries are dispatched only to shards owning relevant chunks no scatter/gather. Should you need to reverse sharding, sh.unshardCollection(&quot;market.trades&quot;) tears down the metadata, and sh.moveCollection(&quot;logs.events&quot;, &quot;shard02&quot;) rebalances unsharded data without downtime. Following shard-aware dispatch, change streams deliver updates in real time.</p><h4>Stage 9: Change Streams &amp; CDC Flow Control</h4><p>Your analytics service subscribes to a change stream. In MongoDB 8.0, <strong>compact resume tokens</strong> reduce wire payloads by ~40 %, and any $match in the stream pipeline pushes down to the oplog reader ensuring only pertinent events traverse the network. If a consumer falls behind by more than 10 MB of buffered events, the server applies back-pressure, pausing oplog forwarding to bound memory usage. Simultaneously, the time-series engine accelerates telemetry workloads.</p><h4>Stage 10: Time-Series Buckets &amp; Query Optimization</h4><p>Per-second CPU telemetry ingests into time-series buckets targeting ~1 MB compressed size. Version 8.0’s adaptive bucketizer dynamically adjusts fill thresholds based on data variance, guaranteeing predictable rollover. Secondary indexes on the meta field leverage prefix-compression and quantile sketches, supplying the query planner with precise cardinality estimates allowing analytics such as “average CPU by host per minute” to execute entirely at the bucket level without full document scans. As night falls, backup and point-in-time recovery ensure data durability.</p><h4>Stage 11: Backup, PITR &amp; Rapid Recovery</h4><p>An on-prem “Atlas-style” backup engine employs a hidden change stream to capture page-level diffs, producing incremental snapshots that reduce RTO to minutes even on multi-petabyte clusters. Continuous Point-In-Time Recovery archives the oplog every 5 s to S3.</p><h4>Stage 12: Monitoring, Alerts &amp; CI-Driven Tuning</h4><p>Throughout this relay, Prometheus scrapes serverStatus(), queryStats(), indexStats(), and your custom UDF-exported history-store metrics. Automated alerts trigger on WT eviction &gt; 500 events/sec, slow operations exceeding 1 % of total queries, or oplog lag &gt; 5 s. Your CI pipeline—powered by YCSB profiles that mimic peak traffic gates every schema, index, and configuration change so that any regression over 10 % in 99th-percentile latencies fails the build. Nightly drift-detection jobs SSH into each mongod, pull the live mongod.conf, diff it against the Git master branch, and auto-file tickets for any deviations.</p><h4><strong>Conclusion</strong></h4><p>By meticulously choreographing each stage zero-copy network ingress, lock-free task scheduling, cached ACL checks, AST parsing and canonical fingerprinting, PlanCache acceleration with multi-plan feedback, SIMD-enhanced SBE execution with cooperative yields.</p><p>WiredTiger’s MVCC caching and bounded journaling, ESR-ordered index scans with pre-splits, majority-committed replication and lean two-phase commits, tag-aware mongos routing, compact change-stream delivery with back-pressure, adaptive time-series bucketing, incremental backups with fine-grained PITR, and continuous telemetry with CI-gated performance benchmarks you construct a living, breathing performance engine.</p><p>MongoDB 8.0’s 2025-grade optimizations ensure deterministic, global sub-millisecond SLAs, and this stage-by-stage blueprint is the definitive guide for engineering enterprise-grade systems with surgical precision.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=267d40d358f5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Enterprise Design: A Holistic Approach to Building Purposeful Endeavors]]></title>
            <link>https://medium.com/@eliehannouch800/enterprise-design-a-holistic-approach-to-building-purposeful-endeavors-6479ef83df66?source=rss-79f152ec915------2</link>
            <guid isPermaLink="false">https://medium.com/p/6479ef83df66</guid>
            <dc:creator><![CDATA[Elie hannouch]]></dc:creator>
            <pubDate>Wed, 15 Jan 2025 21:20:17 GMT</pubDate>
            <atom:updated>2025-01-15T21:20:17.704Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kWwhn0sErPkSyZNRw8fkHA.jpeg" /><figcaption>Image Reference: <a href="https://enterprise.design/">https://enterprise.design/</a></figcaption></figure><p>An enterprise is much more than a legal or financial entity; it is a collective human endeavor driven by shared ambitions. Enterprises bring together people to achieve goals that surpass individual capabilities, encompassing commercial, non-profit, governmental, and ideological purposes. By focusing on collaboration and aligning diverse elements, enterprise design enables organizations to function cohesively and purposefully in an ever-changing ecosystem.</p><h4>Understanding the Enterprise</h4><p>At its core, an enterprise represents the ambitions and activities of people united by a common purpose. These ambitions often extend beyond everyday needs, aiming for transformative outcomes that individuals cannot achieve alone. Enterprises are not static entities; they evolve and adapt as they interact with their ecosystems. While enterprises frequently take formal organizational forms, such as companies, associations, or departments, the enterprise itself transcends these structures. It encompasses people, tasks, and capabilities that together create value, often including elements outside the control of formal organizations but within their sphere of influence.</p><p>This people-centered perspective shifts away from traditional views of enterprises as merely legal or financial entities. Instead, it emphasizes the dynamic interplay of collaboration, purpose, and design. An enterprise’s true power lies in its ability to inspire and enable people to contribute meaningfully toward shared goals.</p><h4>What is Enterprise Design?</h4><p>Enterprise design is the intentional and holistic structuring of an enterprise to align its identity, experience, and architecture. It facilitates collaboration among specialized experts to balance diverse elements and achieve the enterprise’s overarching purpose. By connecting disciplines and fostering co-creation, enterprise design delivers outcomes that are aligned, coherent, and impactful.</p><h4>Key Aspects of Enterprise Design:</h4><h4>Identity</h4><ul><li>Questions Addressed: <strong>Why does the enterprise exist? Who are we? What matters to us?</strong></li><li>Identity encompasses the beliefs, purpose, and shared story of an enterprise. It reflects the values and behaviors of its people, which are conveyed through its brands, content, and positioning in the ecosystem. A strong identity fosters a sense of belonging and alignment among stakeholders, serving as a unifying force that drives collective action.</li></ul><h4>Architecture</h4><ul><li>Questions Addressed: <strong>How does the enterprise operate? What are its capabilities? What makes everything work together?</strong></li><li>Architecture involves the structures, processes, and capabilities needed for the enterprise to function. This includes its internal organization, assets, and connections to the external ecosystem. Effective architecture ensures seamless operations, adaptability to change, and resilience in the face of external challenges. It serves as the backbone of the enterprise’s functionality.</li></ul><h4>Experience</h4><ul><li>Questions Addressed: <strong>What role does the enterprise play in people’s lives? What can it do for them?</strong></li><li>Experience focuses on how the enterprise impacts people through their interactions. This includes the tasks they perform, the journeys they undertake, and the channels through which they connect with the enterprise. These interactions shape the perception of the enterprise and the value it delivers. A positive experience builds trust, loyalty, and long-term engagement.</li></ul><h4>Intersections of Enterprise Design</h4><p>The overlap between the three facets creates intersections, which act as bridges connecting disciplines:</p><ul><li><strong>Organization</strong>: How teams collaborate, define roles, and structure their working architecture. Effective organization fosters efficient communication, accountability, and adaptability within the enterprise.</li><li><strong>Product</strong>: The outputs of the enterprise’s efforts and the value these create for people. Products are the tangible or intangible manifestations of the enterprise’s capabilities, shaping how it delivers on its promises.</li><li><strong>Brand</strong>: The reputation and perception of the enterprise as shaped by people’s experiences. A well-designed brand captures the essence of the enterprise’s identity and conveys it effectively to internal and external audiences.</li></ul><p>By addressing these intersections, enterprise design ensures that isolated disciplines converge to form a cohesive whole, enabling a seamless integration of identity, architecture, and experience.</p><h4>The Importance of Holistic Alignment</h4><p>Enterprises function best when their elements are intentionally aligned and designed to work in harmony. Misalignment between identity, architecture, and experience can create inefficiencies, disconnects, and lost opportunities. Holistic alignment ensures that every aspect of the enterprise contributes to its overall purpose and enhances its ability to create value for all stakeholders.</p><h4>Conclusion</h4><p>Enterprise design is an evolving practice that integrates identity, experience, and architecture into a holistic framework. It moves beyond analytical and technical expertise to embrace creativity, collaboration, and strategic insight. By aligning its many elements, an enterprise can achieve its purpose and make a meaningful impact on its ecosystem. In a world of constant change, enterprise design empowers organizations to adapt, thrive, and fulfill their ambitions, ensuring that their contributions resonate across the communities they serve. In the next article, we will explore the building blocks of enterprise design in detail.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6479ef83df66" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Mongos: The Orchestrator of MongoDB’s Distributed Sharded Architecture]]></title>
            <link>https://medium.com/@eliehannouch800/mongos-the-orchestrator-of-mongodbs-distributed-sharded-architecture-66860551ccc9?source=rss-79f152ec915------2</link>
            <guid isPermaLink="false">https://medium.com/p/66860551ccc9</guid>
            <category><![CDATA[mongodb]]></category>
            <category><![CDATA[mongo]]></category>
            <category><![CDATA[query-routing]]></category>
            <category><![CDATA[horizontal-scaling]]></category>
            <category><![CDATA[sharding]]></category>
            <dc:creator><![CDATA[Elie hannouch]]></dc:creator>
            <pubDate>Sat, 07 Dec 2024 16:46:52 GMT</pubDate>
            <atom:updated>2024-12-07T16:46:52.151Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/960/0*kv0s6OUBaYhBZqkM" /></figure><p>MongoDB is synonymous with modern scalability, and one of its most powerful features is <strong>sharding</strong>, a distributed approach to handling massive datasets and high-throughput applications. At the heart of this architecture lies <strong>mongos</strong>, the query router that transforms a cluster of independent shards into a unified, high-performance database system.</p><p>While mongos might appear to be a straightforward query router, it is, in reality, the linchpin of MongoDB’s sharded architecture. It handles complex query routing, ensures efficient metadata management, and facilitates fault tolerance, enabling MongoDB to achieve seamless scalability and resilience.</p><p>In this deep dive, we’ll explore mongos’s inner workings, focusing on its role in query lifecycle management, metadata synchronization, advanced aggregation, shard balancing, and fault tolerance. By the end, you’ll appreciate why mongos is the powerhouse behind MongoDB’s distributed database excellence.</p><h3>The Foundation of Sharding: Why Mongos?</h3><p>In horizontally scaled database systems, three key challenges arise:</p><ol><li><strong>Distributing data evenly across nodes.</strong></li><li><strong>Executing queries efficiently in a distributed environment.</strong></li><li><strong>Maintaining consistency and availability.</strong></li></ol><p>MongoDB addresses these challenges through <strong>sharding</strong>, which splits data across multiple servers, called <strong>shards</strong>, using a <strong>shard key</strong>. Each shard handles a specific subset of the data, enabling MongoDB to:</p><ul><li><strong>Scale out</strong> storage and compute capacity as data and traffic grow.</li><li><strong>Maintain high query performance</strong> by dividing workloads across shards.</li><li><strong>Support high availability</strong> through replica sets for redundancy within each shard.</li></ul><p>While sharding enables massive scalability, it introduces a layer of complexity. Clients cannot query individual shards directly without knowing which shard contains the data. This is where <strong>mongos</strong> becomes indispensable. Acting as the central coordinator, mongos:</p><ul><li><strong>Abstracts complexity</strong> by exposing a unified interface to clients.</li><li><strong>Handles query routing</strong> to ensure the right shards are accessed efficiently.</li><li><strong>Manages metadata</strong> to maintain an accurate and dynamic view of the cluster.</li></ul><h3>Query Lifecycle in Mongos</h3><p>The lifecycle of a query in a sharded MongoDB cluster involves several sophisticated steps, each driven by efficient algorithms and metadata management.</p><h3>1. Query Reception</h3><p>When a query is submitted, mongos acts as the <strong>entry point</strong> to the cluster. It:</p><ul><li><strong>Parses the query</strong> to identify filters, projections, and operations (e.g., CRUD or aggregation).</li><li><strong>Decides the routing strategy</strong>: Whether the query is targeted (single shard) or scatter-gather (all shards).</li></ul><h4>Key Tasks:</h4><ol><li><strong>Parsing</strong>: Break down the query into its components to determine shard key involvement.</li><li><strong>Routing Strategy</strong>:</li></ol><ul><li><strong>Targeted Queries</strong>: Queries containing the shard key are routed to the appropriate shard(s), minimizing latency.</li><li><strong>Scatter-Gather Queries</strong>: Queries without shard key involvement or requiring global data (e.g., aggregations) are broadcast to all shards.</li></ul><h3>2. Metadata Lookup and Management</h3><p>Efficient query routing relies on <strong>metadata</strong> stored on <strong>config servers</strong>. This metadata defines the structure of the sharded cluster, enabling mongos to route queries accurately.</p><h4>Metadata Components</h4><ol><li><strong>Shard Key Metadata</strong>: Defines how data is distributed across shards.</li><li><strong>Chunk Metadata</strong>: Tracks shard key ranges and the location of each chunk.</li><li><strong>Cluster Topology</strong>: Maintains information about shard health, connectivity, and availability.</li></ol><h4>How Metadata Lookup Works</h4><p>For <strong>range-based sharding</strong>:</p><ul><li>Mongos performs a <strong>binary search</strong> over the sorted shard key ranges to locate the appropriate shard(s).</li><li><strong>Complexity</strong>: O(log⁡n), where n is the number of ranges.</li></ul><p>For <strong>hashed sharding</strong>:</p><ul><li>Mongos computes a consistent hash of the shard key and matches it to a chunk.</li><li><strong>Complexity</strong>: O(1)</li></ul><h4>Caching for Efficiency</h4><p>To reduce dependency on config servers, mongos maintains a <strong>local cache</strong> of metadata:</p><ul><li><strong>Periodic Refresh</strong>: Metadata is updated periodically or when cluster topology changes.</li><li><strong>Fallback</strong>: In case of temporary config server unavailability, mongos serves queries using cached metadata.</li></ul><h3>3. Query Routing</h3><p>Mongos routes queries using one of two strategies, depending on the presence of the shard key:</p><h4>A. Targeted Query Execution</h4><p>For queries that include the shard key:</p><ol><li>Mongos calculates the target shard(s) using metadata.</li><li>The query is routed directly to the relevant shard(s).</li></ol><p><strong>Example</strong>: If the shard key city_id is distributed as:</p><pre>Shard 1: { city_id: [1, 100) }<br>Shard 2: { city_id: [100, 200) }<br>Shard 3: { city_id: [200, 300) }</pre><p>A query for { city_id: 150 } involves:</p><ul><li>A binary search to identify the range [100, 200).</li><li>Routing directly to <strong>Shard 2</strong>.</li></ul><h4>B. Scatter-Gather Queries</h4><p>For queries without a shard key:</p><ol><li>Mongos broadcasts the query to all shards.</li><li>Each shard processes the query locally.</li><li>Mongos merges the results into a unified dataset.</li></ol><p><strong>Algorithm: K-Way Merge Sort</strong></p><ul><li>For sorting operations, mongos uses a <strong>k-way merge sort</strong> to combine sorted results from k shards.</li><li><strong>Complexity</strong>: O(m log⁡ k), where m is the total number of elements.</li></ul><h3>4. Advanced Aggregations</h3><p>MongoDB’s aggregation framework in a sharded cluster uses a <strong>two-phase execution model</strong>:</p><ol><li><strong>Shard Phase</strong>:</li></ol><ul><li>Early pipeline stages (e.g., $match, $group) are pushed to shards for local execution.</li><li>Shards return partial results to mongos.</li></ul><p><strong>2. Merge Phase</strong>:</p><ul><li>Mongos combines partial results and executes global stages (e.g., $sort, $limit).</li></ul><h4>Optimization Techniques</h4><ol><li><strong>Pipeline Optimization</strong>:</li></ol><ul><li>Push as many stages as possible to shards, minimizing inter-shard communication.</li></ul><p><strong>2. Reduced Data Transmission</strong>:</p><ul><li>Shards only send aggregated or filtered results to mongos, reducing network overhead.</li></ul><pre>db.orders.aggregate([<br>  { $match: { region: &quot;EMEA&quot; } },<br>  { $group: { _id: &quot;$product&quot;, totalSales: { $sum: &quot;$sales&quot; } } },<br>  { $sort: { totalSales: -1 } },<br>  { $limit: 5 }<br>]);</pre><ul><li>$match and $group execute locally on shards.</li><li>$sort and $limit are applied globally by mongos.</li></ul><h3>5. Balancing Data Across Shards</h3><p>Over time, data distribution may become uneven, creating hotspots. MongoDB’s <strong>balancer process</strong> redistributes chunks to ensure even utilization.</p><h4>Chunk Migration</h4><ol><li><strong>Identify Imbalance</strong>: The balancer monitors shard utilization metrics (e.g., chunk count, storage size).</li><li><strong>Select Chunk</strong>: A chunk from an overutilized shard is chosen for migration.</li><li><strong>Two-Phase Migration</strong>:</li></ol><ul><li><strong>Phase 1</strong>: Copy the chunk to the target shard while queuing writes.</li><li><strong>Phase 2</strong>: Update metadata in config servers and delete the chunk from the source shard.</li></ul><h4>Consistency Mechanism</h4><p>MongoDB uses a <strong>two-phase commit protocol</strong> to ensure:</p><ul><li>Atomicity: Migration is either fully completed or fully rolled back.</li><li>Consistency: Prevents split-brain scenarios where multiple shards claim ownership of a chunk.</li></ul><h3>6. Fault Tolerance and Resilience</h3><p>Mongos and the sharded cluster are engineered for high availability:</p><ol><li><strong>Redundant Mongos Instances</strong>: Multiple mongos instances distribute traffic and provide failover.</li><li><strong>Replica Sets for Shards</strong>: Each shard is a replica set, ensuring data redundancy.</li><li><strong>Config Server Resilience</strong>: Config servers operate as a replica set to prevent single points of failure.</li></ol><h4>Network Partitions and Failures</h4><ul><li>Mongos retries queries and writes on reachable replicas.</li><li>Partial results are served during shard unavailability, enabling <strong>eventual consistency</strong>.</li></ul><h3>Conclusion</h3><blockquote>Mongos is much more than a query router — it’s the <strong>orchestrator</strong> of MongoDB’s distributed architecture. From query routing to metadata management, shard balancing, and fault tolerance, mongos ensures MongoDB scales effortlessly while maintaining consistency and performance.</blockquote><blockquote>Understanding mongos’s inner workings empowers developers and architects to build robust, scalable distributed systems with MongoDB. It’s the unsung hero that transforms distributed complexity into operational simplicity.</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=66860551ccc9" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>