<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Suresh Eswaran on Medium]]></title>
        <description><![CDATA[Stories by Suresh Eswaran on Medium]]></description>
        <link>https://medium.com/@suresheswaran?source=rss-8a80fb083da1------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*xHNAxa1-lX0A585cid6hWA.jpeg</url>
            <title>Stories by Suresh Eswaran on Medium</title>
            <link>https://medium.com/@suresheswaran?source=rss-8a80fb083da1------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 15:58:15 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@suresheswaran/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Context Engineering is about Structure, Not Retrieval — Part 3: where to start]]></title>
            <link>https://suresheswaran.medium.com/context-engineering-is-about-structure-not-retrieval-part-3-where-to-start-58bb2cbd6f6e?source=rss-8a80fb083da1------2</link>
            <guid isPermaLink="false">https://medium.com/p/58bb2cbd6f6e</guid>
            <dc:creator><![CDATA[Suresh Eswaran]]></dc:creator>
            <pubDate>Sun, 04 Oct 2026 02:43:44 GMT</pubDate>
            <atom:updated>2026-10-04T02:43:44.212Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AKjGvc7gH4q5If6XUrmllw.png" /></figure><p>In part 1 &amp; Part 2 I had argued that structure is what’s missing, and showed what it looks like when it reaches the model — a small resolved subgraph rather than a pile of passages.</p><p>Now the real question is where does the structure come from</p><p>There are many different approaches to address the question and choosing the right approach is important. The choice matters on the approach then</p><p>There’s no shortage of answers to this, and most of them are presented as though they were the only one. They aren’t. There are roughly four ways teams build this, they fail in different ways, and the choice matters more than which graph database you pick.</p><h4>Where does Structure come from?</h4><p>There are about fourways teams build structure for the context from and each works on some conditions and fails on certain conditions.</p><p><strong>Ontology first </strong>— In this appraoch we start with modelling the domain properly . For example finance is not starting with a blank sheet but using an industry standard like BIAN or FIBO, FIHR for medical. The approach is rigorous, and it produces something coherent.</p><p>It fails by never shipping. the model is never quite finished, every edge opens a definitional debate and and eighteen months in there’s a defensible ontology and no working traversal. The failure isn’t that the modelling was wrong. It’s that nothing forced it to stop.</p><p><strong>Extraction-first.</strong> Point a language model at the document corpus and pull out entities and relationships. It’s fast, it produces something within days, and for unstructured material it’s the only option that scales.</p><p>It fails on consistency. The same correspondent comes out four ways across four documents. Relationship types proliferate — routedThrough, routed_via, sentThrough — because nothing constrains the vocabulary. You get a graph that looks impressive and can&#39;t be traversed reliably, which is worse than no graph, because it&#39;s confidently wrong.</p><p><strong>Join over existing masters.</strong> The entities mostly exist already. The counterparty register <em>is</em> an entity master. An CMDB will have the master for applications and its dependencies. The ledger has keys. The contract store references parties that are already identified somewhere. In this approach the graph isn’t a new store of truth — it’s an identity and join layer over systems you already own and already govern.</p><p>This is usually the right place to start in a bank, and it’s the one most often skipped, because it produces no new data and doesn’t look like a project. It fails where the masters are genuinely incomplete, and it inherits whatever quality problems those systems have.</p><p><strong>Query-driven.</strong> Start from the questions the agent has to answer. Model only the entities and relationships those traversals require. Grow when a new question needs something the graph doesn’t have.</p><p>It fails by building something too narrow to extend — you can end up with a structure shaped around ten questions that doesn’t generalise to the eleventh. The discipline it enforces is still worth more than what it costs.</p><h4>What I’d do first ?</h4><p>Those four aren’t alternatives to choose between. They’re stages, and the order matters more than the choice.</p><p>Start with the “Query-driven” approach first. Write down ten questions that the agent needs to answer, not broad categories but actual questions. “like what are the dependencies for this particular applicaitons ?” or “A payment has failed what do I do?” . These questions should be able to address the poblem of which entities matter and which relationships have to exist.</p><p>Then resolve identity over anything else. Take the master entity types that touch the questions. In our earlier example Corrospondent, client and agreement. This stage helps us identify the ownership of the entity , what key resolves them and who decides and owns this.</p><p>The third step is join over what is already known. Once the graph is established hydrating the graph to answer additional questions. The graph at this stage holds almost no data of its own — it holds identities and the edges between systems that already govern their own content. That’s deliberate. Every fact you copy into the graph is a fact you now have to keep current.</p><p>Then extract, narrowly, Contracts and correspondence hold relationships that exist nowhere structured, and extraction is the only way to get at them. But point it at specific relationship types you’ve already defined — not at the corpus with an instruction to find what’s interesting. Extraction populates a vocabulary; it shouldn’t invent one.</p><p>Then let it grow against real questions. The eleventh question will need something the graph doesn’t have. That’s the signal to extend, and it’s a better signal than any roadmap, because it comes with evidence that someone needed it.</p><p>This sequence helps with creating a context that is usable, and does not help answer questions that no agent wants, and establish clear ownership and authority for the context and helps resolve “your context Vs My context” problem.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xOeVETZkMjuhk7K_NwsuTQ.png" /></figure><p><strong>Making it repeatable</strong></p><p>A graph that gives different answers to the same question on different days is worse than no graph.</p><p><strong>Traversal rules.</strong> Left alone, a traversal expands. From the payment you reach the correspondent, from there its parent group, from there every other client that group touches — and now the context window holds half the bank. So the traversal needs limits: how many hops, along which relationship types, in which direction. Those limits are part of the design, not a tuning parameter. A traversal that can expand indefinitely isn’t a selection, it’s a crawl.</p><p><strong>Invariants.</strong> Rules the graph has to satisfy at all times. A payment settles for exactly one client. A correspondent is located in exactly one jurisdiction at a time. An agreement has a start date that precedes its end date. These catch the structural errors that would otherwise surface as a confident wrong answer, and they catch them at write time rather than at query time.</p><p>There’s a limit here worth naming, because it matters more than it first appears. Invariants encode a view of the domain — and where two domains hold different views, writing an invariant means choosing one of them. Held at the level of identity and structural integrity, that’s safe. Held at the level of business meaning, you may be enforcing one domain’s semantics on another without anyone deciding to. That’s a boundary question, and it’s part 4.</p><p><strong>Snapshots.</strong> Version the graph, so a traversal can be replayed against the state it actually ran on. Without this you can’t reproduce a decision an agent made last month, which in a regulated setting means you can’t defend it. It’s also what makes testing possible — a test against a moving graph tests nothing.</p><p>Together these turn the question from “retrieve the most relevant passages” into<em> “</em>retrieve the correct subgraph” — and the word doing the work is correct. Relevance is a matter of degree and you can tune toward it. Correctness is binary, and it’s checkable.</p><p><strong>Knowledge that grows as the work gets done</strong></p><p>Everything so far describes a graph built from what the organisation already knows — source systems, contracts, regulatory text. Modelled once, maintained thereafter.</p><p>But agents working real cases learn things that aren’t in any of those. This correspondent is consistently slow to confirm. This client always asks for clarification before authorising a retry. This corridor fails more often at month-end than the rest of the month combined.</p><p>Today that knowledge dies with the session. Maybe it lands in a summary nobody queries, or in an analyst’s head, which is where it has always lived.</p><p>Written back as edges, it’s available to every traversal that follows. That’s the compounding property worth having: the structure improves as the work gets done, not only when someone schedules more modelling.</p><p>But this needs a boundary, and it’s the part that goes wrong if you skip it.</p><p>An observation is not a fact. “This corridor fails at month-end” is a pattern across fourteen cases in September, not a property of the corridor. If agents write findings back unchecked, the graph fills with plausible assertions that nobody can trace and the next agent treats them with the same confidence as the jurisdiction’s reporting threshold. That undoes the thing the structure was for.</p><p>So observations go in a separate tier. They carry what produced them — how many cases, over what period, by which runs — and they stay marked as observed until a person or a rule promotes them. The provenance machinery from part 2 is already there; this is a second use for it.</p><p>The interesting question isn’t technical. It’s who gets to promote an observation to a fact. In most organisations nobody owns that decision today, because until now there was nothing to decide. That’s part 4.</p><p><strong>Where to start</strong></p><p>If there’s one thing to take from this part, it’s that the hard problems aren’t the ones that look hard.</p><p>The graph database is a procurement decision. The ontology is real work but it’s bounded, and a lot of it already exists. What determines whether this succeeds is narrower and less interesting to talk about: did you resolve identity before you built anything on top of it, did you let the questions constrain the model, and did you keep observations separate from facts.</p><p>None of that is technically difficult. All of it requires someone to decide, and to keep deciding as the systems underneath change.</p><p>Which is where this runs into the thing I’ve been deferring for three parts. Everything here assumes the domain has a settled view of itself — that there’s an answer to what a counterparty is, which system is authoritative for a client, whether an observation has earned the status of fact. In most enterprises there isn’t. Operations and finance can hold genuinely different views of the same entity, both correct for their own purpose, and neither of them wrong.</p><p>That isn’t a modelling problem with a modelling solution. It’s a question about boundaries and ownership, and it’s the last part.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=58bb2cbd6f6e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Context Engineering is about Structure, not retrieval- Part 2: What goes in the context window ?]]></title>
            <link>https://suresheswaran.medium.com/context-engineering-is-about-structure-not-retrieval-part-2-what-goes-in-the-context-window-52e92b97d013?source=rss-8a80fb083da1------2</link>
            <guid isPermaLink="false">https://medium.com/p/52e92b97d013</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[ontology]]></category>
            <category><![CDATA[enterprise-ai-strategy]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[context-engineering]]></category>
            <dc:creator><![CDATA[Suresh Eswaran]]></dc:creator>
            <pubDate>Tue, 01 Sep 2026 23:13:15 GMT</pubDate>
            <atom:updated>2026-09-01T23:13:15.281Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8dbgV4LcVtN9OtzcW0tFLQ.png" /></figure><p>This is the second part of the series, In the first part of the series I explained about the retrieval isn’t missing from most agent systems — a structure is. In the payment example we saw in part 1, where the payment fails at a bank and the question of hat it touched has no answer in any single document. It lives in joins between the ledger, counter-party register and the contract store. The retrival from all the four systems will provide detached insights with nothing connecting them.</p><p>That left a question open. If the structure has to be built rather than retrieved, what does the agent actually get to see?</p><p>Not the whole graph. A model can only work with what fits in its context window, so something has to decide which small part of the graph is relevant to this question, right now.</p><h3>What context produces</h3><p>If we go back to the failed payment example, The question that the agent has is narrow, what is the cause of the payment, is it reportable or shoule we communicate to the customer and what is the next best action the agent shoudl take ?</p><p>A retrieval approach answers this by finding pockets of text about payment failures , reporting obligaiton , reasonable text and none about the payment.</p><p>A graph provides something different, It starts at one node — the failed payment — and follows the edges that the question requires. Where did it route? Who does it settle for? Where is that correspondent located? What does that jurisdiction impose? What does the client’s agreement say? it also provides the next best action in the traversal path of the graph also also the systens and screens to look for in the process.</p><p>Four hops later it has this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Hv_85LMi2mNJYPprIqEpAw.png" /></figure><p>Six nodes and six edges. That is the entire context payload for this question.</p><p>Notice what the edges are doing, Instead of just handing over four passages to the model to decide, it connects them with a stated fact about how these different isolated passages of informaiton are interconnected and helps the agent to reason and act.</p><p>Notice also what the nodes are not. The correspondent node is a BIC and nothing else. No address, no onboarding date, no relationship manager, no risk rating. All of that exists in the counterparty register, and none of it is in the window, because none of it bears on whether this delay is reportable.</p><p>That’s the inversion. Most systems send records and hope the model works out the relationships. The better move is to send the relationships and attach only the values the traversal actually touched.</p><h3>The identity layer</h3><p>Everything in the diagram depends on the fact that correspondent in the payments ledger and the correspondent in the counterparty register have to be recognised as the same thing.</p><p>For example, The ledger holds a BIC. The register holds a legal entity name and an internal counterparty ID. The contract store references the entity as it was written into the agreement in 2019, possibly before a merger changed the name. Three systems, three ways of referring to one bank, and no shared key between them.</p><p>This is obe of the primary resons why traversal would likely to fail , if the agent is unable to follow an edge from the payment to the correspondent if nothing establishes which correspondent. And the failure is quiet — the agent produces an answer, it just isn’t about the right entity.</p><p>Some of this is genuine data debt. Where standards like LEI or BIC exist and systems didn’t adopt them, someone deferred a cost and it has now arrived. But not all of it. Internal counterparty IDs predate those standards, so there was nothing to defer. And even with clean identifiers, you still have to decide whether the entity named in a 2019 contract and the entity after a merger are the same one <em>for this purpose</em>. That last question isn’t a governance failure. It’s a judgment about the business, and it will still be there after the governance is fixed.</p><p><strong>This is the part of the work that gets underestimated.</strong> Teams budget for the graph database and the ontology, then find that most of the effort is identity: deciding what counts as the same entity, building the resolution, and keeping it correct as the underlying systems change.</p><p>The compensation is that identity is reusable in a way retrieval tuning is not. A better reranker helps one query pattern. Resolving your counterparties once makes every traversal that touches a counterparty possible — including the ones you haven’t thought of yet.</p><p><strong>Edges that know when</strong></p><p>The diagram shows the graph as it is today and many organizations woiuld rush to create a similar graph. That’s rarely the question.</p><p>The payment failed on the fourteenth. The relevant question is what was true on the fourteenth — which agreement governed the client then, which correspondent was in the routing chain then, what the jurisdiction’s threshold was then. Contracts get amended. Correspondents get replaced. Thresholds change when regulation changes.</p><p>A graph that only holds current state will answer confidently about the wrong month. So the edges carry validity periods, and the traversal takes a date: not <em>who governs this client</em>, but <em>who governed this client on the fourteenth of last month</em>.</p><p>This costs something. Every relationship becomes a range rather than a fact, and the ontology gets heavier. Skip it and the system works well in demos and misleads in exactly the situations that matter — investigations, disputes, and anything a regulator will later read.</p><p>The related property is provenance. Each edge can carry where it came from: this routing relationship is asserted by the ledger, this obligation by the compliance register, this agreement by the contract store, along with when it was last confirmed.</p><p>That gives the agent something it otherwise cannot have — the ability to say why. Not just <em>the customer must be informed within four hours</em>, but <em>because this correspondent is in this jurisdiction, per the counterparty register as of last Tuesday, and this jurisdiction imposes that threshold, per the obligations matrix.</em></p><p><strong>In a bank this is the difference between a demo and a deployment.</strong> An answer with no traceable basis is unusable in operations regardless of how often it’s right, because the analyst can’t defend it and the regulator won’t accept it. Provenance isn’t a feature you add for compliance. It’s what makes the output actionable at all.</p><p>It also changes the failure mode. When a retrieval system is wrong, you get a confident sentence and no way to find out why. When a traversal is wrong, you get a path — and a path can be inspected, disputed, and corrected.</p><p><strong>What the agent is actually doing differently now</strong></p><p>The agent isn’t smarter. It’s the same model. What changed is that its context is small, specific, and already resolved — seven nodes rather than twenty retrieved passages, with the relationships given rather than inferred. It spends none of its capacity working out how four documents relate to each other, because that work was done before the window was filled.</p><p>That’s the whole of it. Context engineering isn’t about giving the model more. It’s about giving it less, correctly.</p><p>None of this comes free. The graph has to be built, and someone has to decide what it contains — which entities exist, which relationships are worth stating, how far a traversal is allowed to expand. Those are design decisions, and there’s more than one defensible way to make them.</p><p>That’s part three: how you actually build it.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=52e92b97d013" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Context Engineering Is About Structure, Not Retrieval — Part 1: The wrong question]]></title>
            <link>https://suresheswaran.medium.com/context-engineering-is-about-structure-not-retrieval-part-1-the-wrong-question-9477f5d26019?source=rss-8a80fb083da1------2</link>
            <guid isPermaLink="false">https://medium.com/p/9477f5d26019</guid>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[context-engineering-ai]]></category>
            <category><![CDATA[agentic-rag]]></category>
            <category><![CDATA[enterprise-ai]]></category>
            <category><![CDATA[knowledge-graph]]></category>
            <dc:creator><![CDATA[Suresh Eswaran]]></dc:creator>
            <pubDate>Sat, 15 Aug 2026 01:51:28 GMT</pubDate>
            <atom:updated>2026-08-15T01:51:28.328Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QUP3UouuQSBXELYqmVQwZA.png" /></figure><p>In many of my conversations with clients on AI, one question keeps coming up: <strong>“how would knowledge graphs make AI Agents better ?”</strong> .</p><p>The response is mostly that <em>it gives the agent the information it needs</em>. That’s true, but it doesn’t answer the question. A vector database provides the information the agent needs. A search API provides the information the agent needs. Even a PDF pasted into an agent conversation provides the information the agent needs. If that’s the argument, we’ve described a more expensive way to do what the team already has.</p><p>Reflecting on the question, I’ve many times responded with the weak answer — it’s the default one. But the reframing gives the sharper question: are we finding the right text or are we modeling a domain ?</p><p>Knowledge graphs provide more than information. They provide structure. In this blog series I’ll explore why that structure matters, what context engineering actually involves, and how agents perform better when it’s there.</p><p>This is the first of three parts. Here I want to establish the problem: why the question we usually ask about retrieval is the wrong one. The parts that follow cover what structure actually gives an agent, and what it costs to build.</p><h4>Every student has the book</h4><p>Let’s first explore this concept with an analogy,</p><p>Think of an open book exam. Every student has the same text, same index and ability to flip to any page. Now, imagine the book does not have a content or a chapter section.</p><p>what separates students is the type of question, some are page flipping -summarize page 42, flipping works; the student who has never opened a textbook can open page 42 and answer. <em>Compare this argument with the question, to hold two things at once:</em> Now take a question that asks you to hold two thigs at once: in chapter two with the case study in chapter nine. No page contains that answer. It exists only in the relationship between them, and the only student who can produce it is the one who already knows how the book fits together.</p><p>Retrieval-augmented generation is a good page flipper. Give it a lookup question, it will find the answer fast and summarize it at times better than a human. Many problems are solved with this approach.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LWEcDJt3kXF0mOB76c-4Qg.png" /></figure><p>The question is, <em>“Is it enough to automate only lookups ?”</em></p><p>The student who knows the book built that structure in their own head, over weeks. An agent cannot do this, It arrives at every question with no accumulated sense of how your domain fits together, and it won’t develop one by reading more pages faster. <strong>If the structure isn’t in the data, it doesn’t exist.</strong></p><p>So the work shifts. Instead of asking how to retrieve better, we ask what a well-modeled domain would need to look like for the synthesis questions to become answerable at all.</p><p><strong>What that looks like in practice</strong></p><p>Let’s extend the analogy to payment operations at a bank, where I’d want to put AI agents to work handling failed transactions.</p><p>A payment fails. Someone in operations now has to reconstruct what it touched: which corridor it moved through, which correspondent bank was involved, which client agreements sit behind it, and which of those clients are in jurisdictions where a delay becomes a reportable breach.</p><p>There is no document that answers this. The payments ledger knows the transaction. The counterparty register knows the correspondent. The contract store knows the agreements. The obligations matrix knows the reporting thresholds. The answer exists in the joins between them, and today a person assembles it by hand, against a clock.</p><p>Retrieve from all four systems and you get four plausible passages with nothing connecting them. Tune the retriever and you get better passages, still unconnected. The operations analyst is now doing the same assembly work as before — only with an agent’s output as the starting point instead of the source systems. The gap isn’t in the finding. It’s that nothing in the representation says this ledger entry and this register row refer to the same correspondent, or that this client is exposed to that corridor.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CeDp7xhGm6xXXufRJU_R7w.png" /></figure><p>That’s the missing structure. <strong>Not more context — connected context</strong>.</p><p>Which raises the harder question. If the structure has to be built rather than retrieved, what exactly are we building, and what does an agent do with it that it couldn’t do before? That’s part two.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9477f5d26019" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Patterns for increased resiliency in eco-system banking — Part 1]]></title>
            <link>https://suresheswaran.medium.com/patterns-for-increased-resiliency-in-eco-system-banking-part-1-443f57044108?source=rss-8a80fb083da1------2</link>
            <guid isPermaLink="false">https://medium.com/p/443f57044108</guid>
            <dc:creator><![CDATA[Suresh Eswaran]]></dc:creator>
            <pubDate>Sun, 30 Oct 2022 11:12:51 GMT</pubDate>
            <atom:updated>2022-10-30T11:12:51.954Z</atom:updated>
            <content:encoded><![CDATA[<h3>Patterns for increased resiliency in eco-system banking — Part 1</h3><p>Two part blog series on resiliency patterns in eco-system banking</p><p>In this blog, I would like to talk about some of the patterns that we can adopt while implementing MicroServices architectures to expose API;s to ecosystem partners.</p><p><strong>Defining ecosystem banking</strong> — Many banks have embarked on a continuous journey of business transformation through adoption of API;s. This has enabled banks to provide their services through new channels of engagement through their partners. Thus forming an ecosystem of partners providing banking services to their customers. Some of the most popular use-cases for ecosystem banking are, wallet providers, e-commerce platforms providing payment services, bill payments etc.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GffzZ-gYxfKgywiZBSulCA.png" /></figure><p>API’s have played a crucial role in adoption of ecosystem banking through simplification and integration to banking systems. Covid has accelerated this journey for banks. Agility and time-to-market has been one of the key aspects for ecosystem banking. This has been achieved through 2-speed architecture to de-couple the legacy technology debt and enable acceleration of delivery.</p><p><strong>Challenges that eco-system banking brings</strong> — One of the major challenges that the eco-system banking brings is the changes to workload patterns. The workload patterns changes from seasonal peaks to unpredictable workloads. Earlier, the banks knew the seasonal peaks that they customers were driving(Branch peak hours, pay days etc..), with partners coming in the workloads become unpredictable (ex — e-commerce site may announce a sale).</p><p>The digital layer has the ability for elastic compute through the adoption of MicroServices architecture. Many times the legacy systems are constrained with technology debt and capacity to scale on demand, thus causing resiliency issues leading to service degradation and unavailability.</p><p>In order to overcome these challenges, here are some of the patterns that can be adopted to minimize the resiliency issues and improve end customer satisfaction.</p><p>Pattern 1 — Circuit breaker pattern</p><p>pattern 2 — Bulk head pattern</p><p>pattern 3 — Caching</p><p>Pattern 4— Event driven async integration</p><p>Pattern 5— Autoscaling</p><p>It is important to understand the resiliency issue of the organization to ensure that some of these patters are implemented as part of the design to help address service disruptions while able to react to market changes quickly.</p><h4>Pattern 1 — Circuit breaker</h4><p>Consider a scenario where “service A” is calling another service “Service B”. Due to certain issues service B becomes un-responsive. The pattern looks for a certain error threshold (say the service has been unresponsive for more than 1 minute). In this case the circuit breaker changes the state to open and tries to execute any alternative flows from service A. If none available then service A does not waste resources and sends an error message back to the caller. The circuit breaker keeps monitoring the service B end-point. If the service is responsive then the circuit breaker changes state to closed and the system is back to normal. The service state can also move to an “partially-closed” state to not stress (service B) in this case before going to a fully “closed” state.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nuPnRQUFTheHz4gawjUGcw.jpeg" /><figcaption>Circuit breaker pattern</figcaption></figure><h4>Pattern 2 — Bulkhead pattern</h4><p>Bulkhead pattern extends the idea from cargoships where multiple partitions called bulkheads are used to limit any damage within the partition(bulkheads) and not spread across other partitions.</p><p>Most of the microservices we build are containerised . Each container is allocated limited resources(CPU, memory etc) during runtime. For example any application crash due memory leaks in the code will be isolated within the container and this will not bring down other containers where other services are running.</p><p>Another example of usage of bulkhead pattern is while connecting to any resource using a connection pool specifically attached to a service. In the example below, Service A and Service B are connected to the same backend resource using its own connection pool objects. In case of a surge in number of transactions there is a possibility of connection pool exhaustion, but isolated to service B without impacting Service A or the backend resource.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gpkj4Pv8UWc-qLakzeMm7A.jpeg" /></figure><p>We will cover the remaining patters in part 2 of the blog.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=443f57044108" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why is immutable infrastructure a key architectural decision for cloud adoption?]]></title>
            <link>https://suresheswaran.medium.com/why-is-immutable-infrastructure-a-key-architectural-decision-for-cloud-adoption-2bf806989101?source=rss-8a80fb083da1------2</link>
            <guid isPermaLink="false">https://medium.com/p/2bf806989101</guid>
            <category><![CDATA[cloud-operating-model]]></category>
            <category><![CDATA[immutable]]></category>
            <category><![CDATA[cloud-computing]]></category>
            <category><![CDATA[gitops]]></category>
            <dc:creator><![CDATA[Suresh Eswaran]]></dc:creator>
            <pubDate>Mon, 02 May 2022 11:23:13 GMT</pubDate>
            <atom:updated>2022-05-02T11:28:50.267Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eM899LDlOk9xNXatxndTmg.jpeg" /></figure><p>Some of the questions I usually get asked while having cloud discussions are,</p><blockquote><em>“ how do I make changes to the services or infrastructure deployed in the cloud ?”</em></blockquote><blockquote><em>“how do I apply patches for applications running on the cloud and keep all my environments updated? “</em></blockquote><p>Making changes and upgrading the environments has been a challenge to many enterprises leveraging traditional infrastructure and legacy methods of managing infrastructure operations, thus creating inconsistencies between the environments in the overall enterprise infrastructure landscape. These environment inconsistencies lead to significant technology debt over time and get highlighted as exceptions in management reports. Some of these exceptions are not often addressed due to time constraints, cost, and focus on delivering business priorities; thus, the cost to remediate increases significantly.</p><p>In this blog, I would like to introduce the concept of immutable platforms and how to adopt this to manage changes in cloud environments. Immutable platform refers to the immutability beyond infrastructure to support middleware and application stack and build immutable platforms, including management of full-stack technologies.</p><p>Immutability refers to a state of a resource/service that cannot be changed once deployed. In this case, we recreate the instance every time with the new deployment and changes. In the immutable model, we redeploy the servers/applications/resources with the latest state and never perform an in-place upgrade for the instances. With this approach, there is always consistency across environments and reduces the overhead of managing technology debt and minimizing technology risk. Several organizations have adopted the immutable model through the usage of containers, where the container is patched as a new version, tested, and redeployed again. The immutable platform takes this approach to the traditional applications running on the cloud as VM’s, where we will redeploy the application along with the underlying infrastructure, subsequently decommission the first instance.</p><p>Immutability is not a feature provided by cloud providers, but the organisational change management process and operating model that needs to be established to create the immutable platform.</p><p>The following diagrams explain the difference between mutable model and immutable model. This scenario captures a full-stack instance we deploy, and the operating system requires patching.</p><p>In the mutable model (“Figure 1”), the operating system is patched in the same instance, impacting the availability, resiliency, testing, and ability to roll back.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/820/1*S9RcpbaX_oEmvd57sp62Xg.png" /><figcaption>Fig 1 — Mutable model</figcaption></figure><p>While adopting immutable model, a new instance of the full stack technology is created, tested and rolled out in to production while decommissioning the old instance.In this approach we always rollback the changes due to upgrade failures and this ensures that the system is available for end users during the upgrade process.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/793/1*46q3XLCeGeOH1_fbWcDM2g.png" /><figcaption>Fig 2 — Immutable model</figcaption></figure><p><strong><em>Why is it relevant in the cloud world ?</em></strong></p><p>Capacity and availability of infrastructure are always a challenge in the traditional on-prem, thus creating friction to drive new upgrades and changes without impacting business and end-users while co-existing with the current instances. In the cloud realm, the availability of on-demand environments and the ability to provision instantly enables a mechanism to make necessary changes to the infra, apply patches and modify configurations as separate deployments without impacting the business or end users driving some of the following benefits</p><ol><li><em>Zero down time upgrades</em></li><li><em>Ability to rollback to previous versions incase of failures</em></li><li><em>Consistent and repeatable deployments</em></li><li><em>Simplified patch management and upgrades</em></li></ol><p>The following considerations are required to establish an immutable cloud platform,</p><p><strong><em>Config management — </em></strong>Defining and classifying the configurations that we can change as an in-place instance versus the changes that need to apply through immutable platform.</p><p><strong><em>Change management process</em></strong> — Redefining the processes required to apply changes to the production systems.</p><p><strong><em>GitOps</em></strong> — Automation through gitOps pipeline is crucial and a way to test and certify before applying the changes into production. Creating a platform engineering team to build the necessary automation and baseline required to drive immutable processes.</p><p>As part of this blog we understood the differences between mutable and immutable infrastructure and its benefits; thus, we need to include this as a critical architectural decision for adopting cloud.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2bf806989101" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Ops framework for accelerated cloud adoption]]></title>
            <link>https://suresheswaran.medium.com/ops-framework-for-accelerated-cloud-adoption-ac20cecd9b25?source=rss-8a80fb083da1------2</link>
            <guid isPermaLink="false">https://medium.com/p/ac20cecd9b25</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[operations]]></category>
            <category><![CDATA[cloud-migration]]></category>
            <dc:creator><![CDATA[Suresh Eswaran]]></dc:creator>
            <pubDate>Sat, 15 Jan 2022 09:53:41 GMT</pubDate>
            <atom:updated>2022-01-15T10:59:19.976Z</atom:updated>
            <content:encoded><![CDATA[<p>Cloud adoption is a crucial imperative as part of technology strategy for many organizations. While these organizations are able to adopt cloud for pilot and non-critical workloads as part of their innovation and digital transformation, they are constrained with the ability to accelerate and scale their journey.</p><p>One of the significant challenges for accelerated cloud adoption is using traditional “technology operating models” and governance geared towards running on-Prem platforms, infrastructure, and applications. The policies and processes supporting the conventional operating models focus on establishing controls to help minimize technology risk, address information security, and address compliance.</p><p><strong><em>“Yes, these controls are a must for cloud adoption,”</em></strong> In fact, the cloud transformation needs to be looked at through a more stringent lens. Addressing additional controls creates greater friction to move workloads to the cloud and decelerates the adoption journey.</p><p>This can be addressed by establishing an automation framework that addresses the critical concerns of the cloud COE and the Organization’s policies.<br>The following points address some of the optimizations enterprises can adopt to accelerate their cloud transformation journey.</p><ol><li>Cloud COE — Any large cloud transformation requires a functional COE to be set up. This will cause major organizational restructuring and sometimes derail the progress. In such situations, enterprises can adopt a de-centralized CPE model with shared responsibility and establish new ways of working. A typical COE would address the following concerns</li></ol><ul><li>Technology architecture &amp; governance — This function of the COE establishes architectural patterns and guidelines on how services can be consumed within the organizations. This involves maintaining approved services, introducing new services, lifecycle management, and approved landing zones.</li><li>Infosec — this function establishes guidelines and policies for information security. It provides prescriptive guidelines to classify data and the required requirements for handling it.</li><li>Technology risk management – This function ensures all the risk parameters are evaluated in compliance, regulatory and operational risks. Provides prescriptive guidelines on controls that need to be put in place.</li><li>Operational Concerns — This function will ensure the operating ability for services running on the cloud. This also considers how they should be monitored and ensures that resiliency parameters are included in the design.</li></ul><p>Addressing these concerns with traditional processes will add overhead to run and manage the applications, thus unable to leverage the benefits of the cloud.<br>Automation is vital to mitigate these concerns and reduce the overhead of migrating and managing applications on the cloud. The following DevSecOps framework helps address the problems by automating repeatable processes. The following diagram illustrates a simple view of the Ops framework that will help establish continuous compliance, continuous management &amp; continuous optimization.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/736/1*X2T3aHB4dJWutrk1p7Toew.png" /><figcaption>Ops framework for cloud adoption</figcaption></figure><p>GitOps – The gitops framework is the core to enabling repeatable consistency in service provisioning. The main component of this framework is IaC which encapsulates the guidelines and policies established by the COE as code, namely config-as-code, policy-as-code, security-as-code, ops-as-code . These components are used to orchestrate provisioning. The gitops framework ensures the approved version of code and configuration are used to enable consistency and adherence to the COE guidelines. The overhead of change management and reviews involved can be minimized, thus significantly reducing the SLA’s for cloud service provisioning.</p><p>SecOps – The security-as-code &amp; policy-as-code will provide continuous compliance for deployed services and encapsulate all the required tools and security logs with configuration-driven thresholds for alerting security incidents. This also encapsulates the capabilities needed to track and configuration changes to the running services to detect anomalies.</p><p>AIOps – The Ops-as-code will provide the suitable templates and tools embedded and configured during deployment. These will feed in all the logs to the AIOps engine to ensure observability aspects are considered for cloud services.</p><p>FinOps – This is a paramount focus operational capability that many enterprises overlook. Cloud adoption brings in new financial models that are required adopted through Continuous monitoring of utilization and data-related charges to ensure that the Organization is not over-subscribing and will ensure optimal utilization of cloud services</p><p>Adopting these capabilities with high degree of automation and integrating them to the existing DevOps process will help adopt public-cloud pragmatically, without having to change the organisational structure and ensuring the policies and guides can be addressed.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ac20cecd9b25" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Manage the mess with the mesh — Part 1]]></title>
            <link>https://suresheswaran.medium.com/manage-the-mess-with-the-mesh-part-1-13e2faef46d6?source=rss-8a80fb083da1------2</link>
            <guid isPermaLink="false">https://medium.com/p/13e2faef46d6</guid>
            <category><![CDATA[containers]]></category>
            <category><![CDATA[istio]]></category>
            <category><![CDATA[service-mesh]]></category>
            <category><![CDATA[red-hat]]></category>
            <dc:creator><![CDATA[Suresh Eswaran]]></dc:creator>
            <pubDate>Sun, 15 Aug 2021 06:07:10 GMT</pubDate>
            <atom:updated>2021-08-15T06:07:10.309Z</atom:updated>
            <content:encoded><![CDATA[<h3>Manage the mess with the mesh — Part 1</h3><p>As many large enterprises are moving their workloads to cloud-native microserivces on container platforms, the complexity to manage these MicroServices grows exponentially due to the number of interconnected and chained MicroSerices to enable service orchestration. Some of the key challenges that developers face are</p><ol><li>Service version and canary deployments — Most of the services are built with the base idea of enabling an omni-channel platform. This creates a challenge for enterprises where new features are built for a specific channel and impacts regression for other channels. One of the essential capabilities that is required is to enable service version to support the key channel that has the impact to the feature and rolling out the feature progressively to other channels and customers.</li><li>Complexity of service security in a interconnected MicroService landscape — One of the main gaps of kubernetes platform is to enable a service level security and access control.</li><li>Addressing resiliency challenges — Many of the MicroServices integrate with legacy systems which may not have the same level of scalable architecture. This creates a challenge , when the load increases and the legacy systems are unable to provide responses. In order gracefully handle these at a platform level, a strong pooling and circuit breaker capabilities are required.</li><li>Service traceability— As the chain of service invocation increases, a end to end traceability of the request is required to support the omni-channel services. Ex — to support traceability for multiple steps in a business process.</li></ol><p>These are nfr’s which are required to be implemented across the service landscape . To address these application teams start to write client libraries that are attached to the microservice leading to creation of technical debt and technology dependence and becomes an anti-pattern.</p><p>Netflix OSS used to provide these capabilities as part of their spring cloud framework through Eureka, Ribbon and Hysterix, thus making the framework developer centric. Kubernetes platforms are rich in functions in terms of service discovery and routing , container security but lacked these capabilities focussed on improving developer productivity. Most of these challenges are addressed by Service Mesh , which helps improve developer productivity significantly while offloading some of these capabilities to the Service Mesh.</p><p><strong>Service Mesh</strong></p><p>Service mesh provides a network fabric on top of the Kubernetes platforms to manage connectivity between services offloading the following capabilities from the developer</p><ol><li>Service routing — To enable service version support and canary deployments</li><li>Security — Controlling access to sensitive services, provide controls for access sequence and network security</li><li>Observability — Establishing service traceability of transactions in an inter-connected service-chain</li></ol><p>To enable these, the service-mesh is grouped in to two major components</p><p><em>Control Plane</em> — Management layer that helps with overall management of service mesh (Pilot), routing, collecting telemetry(Mixer) information and security (Citadel).</p><p><em>Data Plane</em> — These are services proxies that are deployed along with the Service as a sidecar container. It acts as a service proxy and intercepts all the request and responses.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/643/1*lGpW607zjZtJptmKATaGXA.png" /></figure><p>In this part we discussed the challenges addressed by Service mesh and overview of Istio Service Mesh, which is one of the most popular and implemented for kubernetes platforms. In the next part we will adress detail some of the challenges and how IstioService Mesh can be implemented to address the same.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=13e2faef46d6" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Implementing Event driven systems for enterprises]]></title>
            <link>https://suresheswaran.medium.com/implementing-event-driven-systems-for-enterprises-c9dfac2244df?source=rss-8a80fb083da1------2</link>
            <guid isPermaLink="false">https://medium.com/p/c9dfac2244df</guid>
            <category><![CDATA[event-hub]]></category>
            <category><![CDATA[kafka]]></category>
            <dc:creator><![CDATA[Suresh Eswaran]]></dc:creator>
            <pubDate>Sat, 03 Jul 2021 04:05:39 GMT</pubDate>
            <atom:updated>2021-07-03T04:05:39.301Z</atom:updated>
            <content:encoded><![CDATA[<p>This is a point of view on best practices to implement Event driven architecture for enterprises</p><p>Many large enterprises are accelerating their digital journey. One of the key tenets of the journey is to understand and use data in real-time to react to customer behaviours and changes. Any new sources of data can be integrated in to the eventing components easily, Some of the key challenges enterprises face implementing event driven systems are</p><ol><li>Legacy systems with lack of integration abilities to generate events</li><li>complex transactions spanning across multiple layers of architecture, each architectural holding a piece to the larger puzzle</li><li>Documented understanding the business process to transaction flows — understanding the customer centric view of the transaction</li><li>Transaction driven design — most of the flows are designed as account, product related and lack customer view</li><li>Batch initiated journeys</li><li>Introducing non-intrusive event generation</li></ol><p>Many event systems can integrate with new sources of data (like social media), enterprise legacy systems hold a lot of data with can provide a wealth of information about their customer behavioural patterns and their journeys enabling</p><ol><li>Better understanding of the customer needs and proactively support the customer</li><li>Triggering services closer to his needs with the potential to convert the need to business</li><li>Ability for businesses to understand his sentiments and help address any concerns</li></ol><p>These attributes in turn help businesses and enterprises to be more proactive and better service the customer thus increasing the customer loyal and retention indirectly translating to new revenue streams and business growth.</p><p>There are two ways to implement an event driven architecture</p><ol><li>Top-down approach — Defining the target set of contextual events that are required to be captured by the data scientists, and working backwards to understand the data flows and sources</li><li>Bottoms-up approach — Stream and push every piece of information to the event streaming engine and let the Data scientists leverage the maze of information available and build the journeys</li></ol><p>Each of the above approach has its own pros and cons, but the key is to choose the right approach based on the following factors</p><ol><li>Cost and complexity for Implementation</li><li>Legacy technology constraints</li><li>More information generated more cost</li><li>Data availability to generate better insights</li></ol><p>In order to overcome the challenges, a pragmatic approach is required to implement event driven systems for enterprises and we talk about the top-down approach</p><ol><li>Data — It is important to understand which customer journeys are must required to generate better insights. Data scientists can help create the journeys and track back the transactions, data sources and data elements. This will help to reduce costs (as every data element that needs to be pushed will incur implementation and technology processing costs)</li><li>Pre-defined Integration patterns — As this platform has to source data from various systems and technologies, its important to understand the key integration patterns on how the source data can be streamed</li><li>Data processing — Creating a simple event processing engine to process the data that is streamed in to business catalogued data in realtime will enable this as the input point for various usecases including data replication</li><li>Event catalog — The data published by the streaming engines can be an extension of Open API’s where third-party partners can subscribe to these events and provide better services to the customer in the wider ecosystems .</li></ol><p>With the introduction of IOT’s and edge devices more and more data will be streamed and enterprises need to be ready with a governed event driven platform to leverage internal data augmented with environmental data about the customer to create better insights, and enabling them to be relevant in the market and ecosystem.</p><p>Though there are number of technologies available in the market , a approach, operating model and governance is required to enable a scaled adoption and avoid the need to handle the deluge of data which is becomes stale and dormant.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c9dfac2244df" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Software architecture and taking inspiration from human physiology]]></title>
            <link>https://suresheswaran.medium.com/software-architecture-and-taking-inspiration-from-human-physiology-9d0660c9b62a?source=rss-8a80fb083da1------2</link>
            <guid isPermaLink="false">https://medium.com/p/9d0660c9b62a</guid>
            <category><![CDATA[software-architecture]]></category>
            <category><![CDATA[support-it-systems]]></category>
            <dc:creator><![CDATA[Suresh Eswaran]]></dc:creator>
            <pubDate>Sat, 26 Jun 2021 15:00:09 GMT</pubDate>
            <atom:updated>2021-06-26T15:09:47.221Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cttna6-93IIvdJtkkUxRTQ.jpeg" /></figure><p>I was watching a documentary on netflix about human body and how it functions. It was fascinating to see how the human body is architected and optimally engineered which has evolved through centuries.</p><ol><li>The human body functions are designed as modular components, they are functionally isolated and can operate without impacting the other functions. Especially while under a threat, it allocates a lot of resources for key body functions to optimally perform under duress, while shutting down other non-critical functions. This makes humans highly resilient. As part of designing IT systems as MicroServices , it is essential to create them as independent functions, so that during heavy stress or increased workloads the underlying resources can be optimally utilized and add them to the critical services required at that point in time.</li><li>The amount of data that human body is able to collect, compress, transmit, process , analyze, optimally persist, create decision flows and create the right response in subseconds is magnificent. The kind of event streaming architecture used is something to learn from. In today’s world where we have a number of data about environment being streamed through IOT devices, transactions, usage patterns and creating new contextual processes(Processes are predefined today) by constantly learning and training the process and eventually becomes core part of the systems.</li><li>The way our brain stores information is optimized, in such a way that it can pull the right information as and when it’s required. It has the ability to archive information with a different SLA’s for retrieval. There is an inherent ability of the brain to classify data based on its usage and able to choose the right persistence layers. Its fascinating that the brain has the ability to use context to choose the right information. It seems like it has a process to context to data mapping and dynamically choose the information it requires based on the context. It’s like a dynamic query generation based on context and process.</li><li>Dynamic parallel processing and contextual resource allocation, is the ability that the brain chooses weather to use parallel processing and if it has the resources to support(like riding a cycle and talking). Where the human has to put effort and thinking to talk and brain seems to be allocating a number more resources to process, while less resources to instructing different muscles to help ride the cycle.</li><li>The SRE function continuously monitors the body functions and sends critical alerts based on thresholds, takes corrective actions and provides constant feedback for continuous improvement.</li></ol><p>Though there are a number of fascinating things that the human body has and it is impossible to recreate them with the technology we have today, we can definitely take inspiration from the way the body is architected and engineered to create better performing and optimal IT systems.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9d0660c9b62a" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>