<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Jorge Nunes on Medium]]></title>
        <description><![CDATA[Stories by Jorge Nunes on Medium]]></description>
        <link>https://medium.com/@jorgenunes10?source=rss-6f61c962a302------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>Stories by Jorge Nunes on Medium</title>
            <link>https://medium.com/@jorgenunes10?source=rss-6f61c962a302------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 13:29:04 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@jorgenunes10/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Integrating Generative AI into Cloud-Native Microservices]]></title>
            <link>https://jorgenunes10.medium.com/integrating-generative-ai-into-cloud-native-microservices-9bded3aef782?source=rss-6f61c962a302------2</link>
            <guid isPermaLink="false">https://medium.com/p/9bded3aef782</guid>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Jorge Nunes]]></dc:creator>
            <pubDate>Sat, 03 Oct 2026 15:38:10 GMT</pubDate>
            <atom:updated>2026-10-03T15:38:10.306Z</atom:updated>
            <content:encoded><![CDATA[<h3>An Architectural Approach to Retrieval-Augmented Generation, Event-Driven Processing, and Scalable AI Services</h3><p><strong>Abstract</strong></p><p>Generative artificial intelligence is increasingly being incorporated into software systems that operate within cloud-native and distributed architectures. Integrating a large language model (LLM) is not solely a model-invocation problem: applications must also address knowledge access, orchestration, latency, workload isolation, security, observability, and failure handling.</p><p>This article presents an architectural analysis of a reference approach for integrating generative AI capabilities into microservices. It examines how retrieval-augmented generation (RAG) can provide external context to models, how event-driven processing can isolate long-running tasks, and how operational controls can reduce — but not eliminate — risks such as unsupported output and data exposure.</p><p>The contribution is analytical rather than experimental. No new benchmark, model-quality score, or production outcome is claimed. Instead, the article identifies architectural decisions, trade-offs, and measurable evaluation dimensions for future implementation.</p><p><strong>Keywords:</strong> Generative AI, Large Language Models, Retrieval-Augmented Generation, Microservices, Cloud-Native Architecture, Event-Driven Systems, Java, AWS, Observability.</p><h3>1. Introduction</h3><p>Integrating generative artificial intelligence into an established application introduces a dependency with characteristics that differ from conventional business services.</p><p>Model inference may have variable response times, consume substantial resources, and return probabilistic outputs rather than deterministic domain results. Furthermore, a model’s training data does not automatically include an organization’s current or private information.</p><p>These properties affect system boundaries, user experience, security controls, and operational design.</p><p>A direct API call to a model may be sufficient for a small prototype. In a distributed application, however, this approach can create tight coupling between user-facing services and an external inference dependency.</p><p>A more deliberate architecture can isolate model access, retrieve relevant domain information when required, and route workloads according to latency and reliability requirements.</p><p>This article addresses the following research question:</p><p><em>How can generative AI capabilities be integrated into cloud-native microservices while preserving service boundaries, managing variable inference workloads, and controlling the risks associated with generated responses?</em></p><h3>2. Technical Background</h3><h3>2.1 Generative Models and LLM Inference</h3><p>Generative models produce new content from an input prompt. Large language models are commonly used for tasks such as summarization, question answering, classification, and text generation.</p><p>In an application architecture, the model is an inference dependency: the application supplies an input and receives a generated output.</p><p>The application remains responsible for authentication, business rules, validation, and deciding whether the output is suitable for downstream use.</p><h3>2.2 Embeddings and Semantic Retrieval</h3><p>An embedding model maps text into a numerical representation that can support similarity search.</p><p>A retrieval pipeline typically processes source documents, divides them into chunks, generates embeddings, and stores vectors alongside text and metadata.</p><p>At query time, the system embeds the user’s question and searches for relevant passages.</p><p>Retrieval quality depends on several factors, including source quality, chunking strategy, metadata, indexing, and ranking choices.</p><h3>2.3 Retrieval-Augmented Generation</h3><p>Retrieval-Augmented Generation combines external information retrieval with model generation.</p><p>A common workflow retrieves relevant passages, adds them to the prompt as context, and asks the model to answer using that context.</p><p>RAG can make application knowledge easier to update than retraining a model for every document change. However, it does not guarantee factual correctness.</p><p>Irrelevant retrieval, incomplete documents, ambiguous prompts, or unsupported model synthesis can still produce unreliable answers.</p><h3>2.4 Event-Driven Processing</h3><p>Event-driven systems communicate through messages or events rather than requiring every operation to complete within a single request-response interaction.</p><p>Queues can buffer work and allow workers to process tasks independently.</p><p>This pattern is relevant to AI tasks such as document ingestion, batch summarization, and large-scale classification, where completion may take longer than an interactive request can reasonably wait.</p><h3>3. Proposed Architecture</h3><p>The reference architecture separates the user-facing API from AI orchestration and model inference.</p><p>A client submits a request to an API Gateway or edge layer, where identity and request limits are enforced.</p><p>A Java/Spring Boot orchestration service validates the request, applies application-level policies, and coordinates retrieval and model calls.</p><p>A retrieval component searches an indexed knowledge base when the use case requires organization-specific context. The model endpoint performs inference, while the orchestration layer formats the response and attaches source references where available.</p><p>These are logical boundaries, not mandatory one-service-per-component deployment instructions.</p><p>A small system may implement orchestration and retrieval in one deployable unit. As workload, ownership, or scaling needs diverge, components can be separated.</p><p>The decision should be driven by operational boundaries and change patterns rather than by a goal of maximizing service count.</p><h3>4. RAG Workflow and Data Lifecycle</h3><p>The RAG workflow has an indexing path and a query path.</p><p>In the indexing path, authorized source content is extracted, normalized, divided into chunks, embedded, and written to a vector index together with metadata and source identifiers.</p><p>In the query path, the user request is embedded, relevant chunks are retrieved and optionally reranked, and a prompt is assembled with the retrieved context. The model generates a response that can be returned with references to the retrieved material.</p><p>Indexing should be treated as a data pipeline, not as a one-time setup task.</p><p>Source updates, deletions, access-control changes, and embedding-model changes can require reprocessing or index maintenance.</p><p>Metadata should preserve provenance and support authorization filtering. Where documents have different access permissions, retrieval must enforce those permissions before their content is included in a prompt.</p><h3>5. Asynchronous AI Processing</h3><p>Not every AI operation should execute synchronously.</p><p>Interactive question answering may require a request-response path with a defined latency budget. By contrast, document enrichment, batch classification, report generation, or large file analysis may be better represented as jobs.</p><p>The API can accept a job, publish a message containing a stable job identifier and minimal required metadata, and return a status reference.</p><p>A worker then retrieves the input, performs the AI workflow, persists the result, and signals completion.</p><p>Queue-based processing can absorb bursts and separate user-facing availability from worker capacity.</p><p>It also introduces distributed-systems concerns:</p><ul><li>Duplicate message delivery.</li><li>Idempotency.</li><li>Poison messages.</li><li>Bounded retries.</li><li>Visibility or lease duration.</li><li>Result-state consistency.</li></ul><p>A job identifier and explicit state machine — such as <em>accepted</em>, <em>running</em>, <em>completed</em>, and <em>failed</em> — make the workflow observable and easier to recover.</p><p>Sensitive prompts and documents should not be copied into messages or logs unless necessary and appropriately protected.</p><h3>6. Reliability, Security, and Observability</h3><h3>6.1 Reliability Controls</h3><p>Model endpoints and retrieval services should be treated as dependencies that can time out, throttle requests, or become temporarily unavailable.</p><p>Timeouts, bounded retries with backoff, concurrency limits, and circuit-breaking behavior can protect upstream services from unbounded waiting and retry amplification.</p><p>Retry policies should distinguish transient transport failures from permanent validation errors.</p><p>When a response is uncertain or incomplete, the application should define whether to return a controlled error, ask the user to retry, or route the task for review.</p><h3>6.2 Security and Data Governance</h3><p>AI integration expands the paths through which application data can flow.</p><p>The design should identify which user inputs, retrieved passages, outputs, and telemetry may be sent to model providers or retained.</p><p>Authentication and authorization must apply to both the API and the retrieval layer.</p><p>Prompt injection, sensitive-data disclosure, unsafe tool invocation, and untrusted retrieved content require explicit threat modeling and layered controls.</p><p>Output validation and human approval are appropriate when generated content can trigger consequential actions.</p><h3>6.3 Observability</h3><p>Traditional service metrics remain useful but are not sufficient to understand an AI workflow.</p><p>Operational telemetry can include:</p><ul><li>Request rate and end-to-end latency.</li><li>Retrieval latency and model inference latency.</li><li>Token usage, where exposed.</li><li>Timeout and throttling rates.</li><li>Queue depth and job age.</li><li>Retry counts and failure rates.</li></ul><p>Traces should connect the user request to retrieval, inference, and asynchronous job stages using correlation identifiers.</p><p>Quality evaluation — such as retrieval relevance, groundedness, citation correctness, and task success — should be measured separately from infrastructure availability.</p><p>Logs should avoid exposing secrets or sensitive prompt content.</p><h3>7. Architectural Trade-Offs</h3><p>No single integration pattern is appropriate for every generative AI application.</p><p>Architectural decisionPotential benefitMain trade-offRAG vs. fine-tuningExternal knowledge can be updated without retraining for every document changeRetrieval and indexing add components and can return irrelevant contextSynchronous vs. asynchronousSynchronous paths support immediate interaction; queues isolate long-running tasksAsynchronous workflows require job state, retries, and completion handlingManaged vs. self-managed inferenceManaged endpoints reduce some infrastructure responsibilitiesProvider constraints, pricing, data handling, and portability must be assessedSingle service vs. decomposed servicesModular design can isolate scaling and ownership boundariesMore services increase deployment, networking, and observability complexityMore context vs. smaller promptsAdditional context may improve coverage for some queriesLarger prompts can increase latency and cost and may add distracting material</p><p>The appropriate choice depends on workload characteristics, reliability requirements, security constraints, and operational capacity.</p><p>Architectural complexity should be introduced when it solves a concrete problem, not simply because a pattern is available.</p><h3>8. Broader Technological Relevance</h3><p>Generative AI services can be applied to knowledge discovery, technical support, document analysis, software engineering assistance, and workflow automation.</p><p>In each case, practical value depends on the fit between the task, available data, quality requirements, and the cost of errors.</p><p>A reusable cloud-native architecture can help teams integrate model capabilities into existing systems without treating the model as a substitute for domain services, access controls, or operational ownership.</p><p>From a systems perspective, the central challenge is not merely generating a plausible response. It is delivering a response that is appropriately grounded, authorized, traceable, and available within the application’s service objectives.</p><p>This makes AI integration a cross-layer engineering problem involving data architecture, distributed processing, security, and evaluation.</p><h3>9. Evaluation Methodology for Future Work</h3><p>The proposed architecture has not been evaluated through a new controlled experiment in this article.</p><p>A future implementation could use a fixed, versioned document corpus and a representative query set to compare retrieval and orchestration configurations.</p><p>The evaluation should report dataset composition, model and embedding versions, chunking and ranking parameters, concurrency, hardware or managed-service configuration, and repeatability conditions.</p><p>Potential measurements include:</p><ul><li>Retrieval precision or recall on labeled queries.</li><li>Answer groundedness assessed against source passages.</li><li>Citation accuracy.</li><li>Task completion rate.</li><li>p50, p95, and p99 latency.</li><li>Token consumption and cost per completed task.</li><li>Queue wait time.</li><li>Failure rate and recovery behavior.</li></ul><p>Results should be reported with the evaluation method and limitations; no single metric captures overall system quality.</p><p>Human review may be required for subjective or high-impact tasks.</p><h3>10. Limitations</h3><p>This article is an architectural analysis and does not present a deployed implementation, performance benchmark, or measured improvement in model quality.</p><p>The diagrams describe logical components and do not prescribe a specific AWS service mapping.</p><p>Model behavior, provider features, prices, and security controls vary by model and deployment environment and should be verified during implementation.</p><p>RAG reduces dependence on information stored only in model parameters but cannot eliminate hallucinations, retrieval errors, data governance risks, or the need for evaluation.</p><h3>11. Future Work</h3><p>Future work can implement a Java/Spring Boot reference service with a documented RAG pipeline and an event-driven ingestion worker.</p><p>Experiments could compare chunking strategies, retrieval methods, reranking, context limits, and synchronous versus queued execution under controlled load.</p><p>Additional work should evaluate authorization-aware retrieval, prompt-injection defenses, model/provider fallback behavior, tracing across asynchronous boundaries, and the operational cost of alternative deployment choices.</p><h3>12. Conclusion</h3><p>Integrating generative AI into cloud-native applications requires more than connecting an API to a language model.</p><p>A maintainable design separates request handling, retrieval, inference, and long-running work according to their responsibilities and operational characteristics.</p><p>RAG can provide application-specific context, while event-driven processing can isolate tasks that do not fit an interactive request path.</p><p>These patterns introduce their own complexity and must be paired with access control, bounded failure handling, observability, and explicit quality evaluation.</p><p>The architectural approach presented here offers a foundation for implementation and measurement, while leaving performance and quality claims to future reproducible experiments.</p><h3>References</h3><p>[1] M. Shah, N. Fort, and R. Muralidhar, “Retrieval Augmented Generation options and architectures on AWS,” AWS Prescriptive Guidance.<br><a href="https://docs.aws.amazon.com/prescriptive-guidance/latest/retrieval-augmented-generation-options/introduction.html">https://docs.aws.amazon.com/prescriptive-guidance/latest/retrieval-augmented-generation-options/introduction.html</a></p><p>[2] Amazon Web Services, “Understanding Retrieval Augmented Generation,” AWS Prescriptive Guidance.<br><a href="https://docs.aws.amazon.com/prescriptive-guidance/latest/retrieval-augmented-generation-options/what-is-rag.html">https://docs.aws.amazon.com/prescriptive-guidance/latest/retrieval-augmented-generation-options/what-is-rag.html</a></p><p>[3] Amazon Web Services, “Retrieval Augmented Generation,” Amazon SageMaker AI Developer Guide.<br><a href="https://docs.aws.amazon.com/sagemaker/latest/dg/jumpstart-foundation-models-customize-rag.html">https://docs.aws.amazon.com/sagemaker/latest/dg/jumpstart-foundation-models-customize-rag.html</a></p><p>[4] Amazon Web Services, “Designing serverless AI architectures,” AWS Prescriptive Guidance.<br><a href="https://docs.aws.amazon.com/prescriptive-guidance/latest/agentic-ai-serverless/designing-serverless-ai-architectures.html">https://docs.aws.amazon.com/prescriptive-guidance/latest/agentic-ai-serverless/designing-serverless-ai-architectures.html</a></p><p>[5] Amazon Web Services, “Generative AI Lens — AWS Well-Architected Framework.”<br><a href="https://docs.aws.amazon.com/wellarchitected/latest/generative-ai-lens/">https://docs.aws.amazon.com/wellarchitected/latest/generative-ai-lens/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9bded3aef782" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Event-Driven Microservices with Apache Kafka]]></title>
            <link>https://jorgenunes10.medium.com/event-driven-microservices-with-apache-kafka-b0c4fdc676e5?source=rss-6f61c962a302------2</link>
            <guid isPermaLink="false">https://medium.com/p/b0c4fdc676e5</guid>
            <category><![CDATA[event-driven-architecture]]></category>
            <category><![CDATA[distributed-systems]]></category>
            <category><![CDATA[cloud-native]]></category>
            <dc:creator><![CDATA[Jorge Nunes]]></dc:creator>
            <pubDate>Wed, 30 Sep 2026 22:49:01 GMT</pubDate>
            <atom:updated>2026-09-30T22:49:01.203Z</atom:updated>
            <content:encoded><![CDATA[<h3>Architectural Patterns for Scalable Event Streaming with Java and Spring Boot</h3><h3>Abstract</h3><p>Event-driven microservices require communication mechanisms that decouple producers from consumers while supporting independent processing, horizontal scaling, and controlled recovery. Apache Kafka addresses this problem through a distributed event-streaming model based on topics, partitions, consumer groups, and offsets.</p><p>This article presents an architectural analysis of Kafka within a Java and Spring Boot microservice environment. The reference laboratory associated with the work uses an Order Service to publish OrderCreated events to Kafka and an Analytics Service to consume those events. The laboratory also combines Kafka with Amazon SNS and Amazon SQS for operational service communication, providing a concrete context for comparing queue-oriented messaging with event-streaming semantics.</p><p>The contribution of this work is architectural rather than a performance benchmark. No new throughput, latency, or recovery measurements are claimed in this article. Instead, the implementation is used as a reproducible reference model for analyzing partition-based parallelism, consumer-group coordination, offset management, event replay, and the operational trade-offs introduced by event streaming. Planned experiments are explicitly identified as future work.</p><p>The analysis highlights a central principle of distributed-system design: Kafka does not merely transport messages between services. Its topic, partition, consumer-group, and offset abstractions influence how services scale, recover, and reason about event history.</p><p><strong>Keywords:</strong> Apache Kafka, event-driven architecture, microservices, event streaming, distributed systems, consumer groups, partitions, offsets, replay, Java, Spring Boot.</p><h3>1. Introduction</h3><p>Microservice architectures distribute application responsibilities across independently deployable components. This decomposition can improve autonomy and enable teams to evolve services independently, but it also transforms communication into a distributed-systems problem.</p><p>Synchronous request/response interactions create temporal coupling. A producer generally depends on a downstream service being reachable and responsive at the time the request is made. As the number of services increases, this coupling can contribute to latency propagation, cascading failures, and coordination complexity.</p><p>Event-driven communication changes this model. Instead of requiring a producer to invoke a consumer directly, a service can publish an event representing a state transition or business fact. Consumers can process that event independently according to their own execution model.</p><p>Apache Kafka extends this idea into an event-streaming model. Rather than treating a message only as work that should be consumed and removed, Kafka organizes records into topic partitions and tracks consumer positions through offsets. Consumer groups provide a mechanism for distributing partition processing across multiple consumer instances.</p><p>The reference laboratory accompanying this article implements a concrete version of this model. The Order Service publishes OrderCreated events to Kafka, while the Analytics Service consumes the event stream. The repository also contains SNS/SQS-based operational paths, making it possible to analyze Kafka as a complementary mechanism rather than as a universal replacement for queues. The repository explicitly identifies Kafka event streaming, consumer groups, and replay-oriented processing as part of the laboratory architecture.</p><p>The central research question is:</p><blockquote><strong><em>How can Apache Kafka be integrated into an event-driven microservice architecture to support scalable event processing, consumer independence, and event replay while keeping architectural and operational complexity manageable?</em></strong></blockquote><h3>2. Technical Background</h3><h3>2.1 Event-Driven Architecture</h3><p>In an event-driven architecture, components communicate through events representing facts that have occurred.</p><p>An event such as:</p><pre>OrderCreated</pre><p>does not necessarily represent a direct command to a specific downstream service. Instead, it can represent a business occurrence that multiple independent consumers may interpret.</p><p>This distinction is important because it separates the producer from knowledge of downstream processing.</p><p>A producer can publish an event without requiring every interested service to execute synchronously.</p><p>The resulting architecture introduces eventual consistency and shifts part of system coordination from direct calls to asynchronous state propagation.</p><h3>2.2 Event Streaming</h3><p>Queue-oriented messaging and event streaming solve related but different problems.</p><p>A queue-oriented model commonly emphasizes work distribution:</p><pre>Producer<br>    |<br>    v<br> Queue<br>    |<br>    v<br>Consumer</pre><p>The event-streaming model used by Kafka emphasizes an ordered log of records organized into topics and partitions:</p><pre>Producer<br>    |<br>    v<br> Kafka Topic<br>    |<br>    +---- Partition 0<br>    |<br>    +---- Partition 1<br>    |<br>    +---- Partition 2<br>    |<br>    v<br>Consumer Groups</pre><p>Kafka’s partition model is particularly relevant to scalability. Partitions provide units over which records can be distributed and consumed in parallel. Kafka documentation identifies partition count as a factor in the maximum consumer parallelism available for a topic.</p><h3>2.3 Topics and Partitions</h3><p>A Kafka topic is divided into one or more partitions. Each partition is an ordered sequence of records.</p><p>A simplified representation is:</p><pre>Partition 0: E0 → E3 → E6<br>Partition 1: E1 → E4 → E7<br>Partition 2: E2 → E5 → E8</pre><p>The partition is therefore more than a storage detail.</p><p>It is an architectural unit that influences:</p><ul><li>ordering;</li><li>parallelism;</li><li>consumer assignment;</li><li>scaling;</li><li>workload distribution.</li></ul><p>Increasing partition count can increase potential consumer parallelism, but it also introduces design considerations around partition keys, ordering requirements, data distribution, and operational management.</p><p>Partition count should therefore be treated as an architectural decision rather than simply as a performance-tuning parameter.</p><h3>2.4 Consumer Groups</h3><p>A consumer group provides a logical processing identity for a set of consumer instances.</p><p>Kafka uses group membership and partition-assignment mechanisms to distribute work among instances in the same group.</p><p>A useful conceptual model is:</p><pre>Kafka Topic<br>     |<br>     v<br>Partitions<br>     |<br>     v<br>Consumer Group<br>     |<br>     +---- Consumer A<br>     |<br>     +---- Consumer B<br>     |<br>     +---- Consumer C</pre><p>The group.id property identifies the consumer group in the standard Kafka consumer model.</p><p>If a topic has more partitions than active consumers, a consumer can be assigned multiple partitions.</p><p>If there are more consumers than partitions, some consumers will not have partitions assigned for that topic.</p><p>This relationship creates an important scalability constraint:</p><blockquote><strong><em>Consumer parallelism is bounded by available partition parallelism.</em></strong></blockquote><h3>2.5 Offsets and Consumer Position</h3><p>Kafka consumers track their position in the event stream using offsets.</p><p>A consumer’s position determines which records it has processed or is expected to process next.</p><p>This makes recovery different from a simple queue-delete model.</p><p>The event remains part of the topic according to its retention and storage configuration, while the consumer maintains its own processing position.</p><p>Kafka’s consumer configuration includes mechanisms such as auto.offset.reset for defining behavior when a group has no valid initial offset or when its current offset is no longer available.</p><p>Offset management therefore becomes part of application architecture rather than merely an infrastructure setting.</p><h3>2.6 Replay</h3><p>Replay is one of the architectural consequences of retaining an event stream.</p><p>A consumer can potentially process historical records again by starting from an earlier offset or by using a new consumer group.</p><p>Kafka documentation describes mechanisms for resetting consumer positions, while Kafka Streams also provides mechanisms for resetting applications and reprocessing input data.</p><p>Replay should not be confused with duplicate delivery in a queue.</p><p>Replay is an intentional change in consumer position.</p><p>Duplicate delivery, by contrast, is an additional delivery attempt of a message that the application may already have processed.</p><p>This distinction is important when designing idempotent consumers and event-driven state reconstruction.</p><h3>3. Proposed Architecture</h3><p>The reference architecture extends the first laboratory by separating operational messaging from analytics-oriented event streaming.</p><p>The core Kafka path is:</p><pre>Order Service<br>      |<br>      | OrderCreated<br>      v<br>Kafka Topic<br>      |<br>      v<br>Analytics Service</pre><p>The complete laboratory contains three asynchronous communication patterns:</p><ol><li>SQS for point-to-point work distribution.</li><li>SNS → SQS for event fan-out to independent operational consumers.</li><li>Kafka for event streaming and analytics-oriented processing.</li></ol><p>This combination is architecturally significant because it avoids treating all asynchronous communication as the same problem.</p><p>Operational work can be represented as queue-based processing, while event streams can remain available for independent consumers and future replay-oriented processing.</p><p>The reference laboratory uses Java 21, Spring Boot, Docker Compose, Apache Kafka, and LocalStack for local development. Kafka is exposed locally on port 9092 in the repository configuration.</p><h3>3.1 Event Model</h3><p>The laboratory uses an OrderCreated event containing application-level identifiers such as eventId and orderId.</p><p>A representative structure is:</p><pre>{<br>  &quot;eventId&quot;: &quot;unique-event-id&quot;,<br>  &quot;type&quot;: &quot;OrderCreated&quot;,<br>  &quot;orderId&quot;: &quot;order-id&quot;,<br>  &quot;customerId&quot;: &quot;customer-id&quot;,<br>  &quot;amount&quot;: 149.90,<br>  &quot;occurredAt&quot;: &quot;timestamp&quot;<br>}</pre><p>The event represents a business occurrence rather than a direct invocation of the Analytics Service.</p><p>The event identifier is particularly important for downstream processing because event-streaming architectures must distinguish event identity from delivery attempts and consumer position.</p><h3>3.2 Processing Boundary</h3><p>The producer is responsible for publishing the event.</p><p>Kafka is responsible for storing and distributing the event according to topic and partition semantics.</p><p>The consumer is responsible for interpreting and processing the event.</p><p>This creates three distinct architectural boundaries:</p><pre>Producer<br>   |<br>   | Event publication<br>   v<br>Kafka<br>   |<br>   | Event distribution<br>   v<br>Consumer<br>   |<br>   | Business processing<br>   v<br>Application State</pre><p>The separation allows each component to evolve independently while preserving an explicit communication contract.</p><h3>4. Kafka Processing Model</h3><h3>4.1 Producer</h3><p>The producer publishes records to a Kafka topic.</p><p>In the laboratory, the Order Service is the producer.</p><p>Its responsibility is to create the OrderCreated event and publish it to Kafka.</p><p>The producer should not need direct knowledge of the Analytics Service’s runtime state.</p><p>This reduces temporal coupling between the operational transaction and downstream analytics processing.</p><h3>4.2 Topic and Partition</h3><p>After publication, the event belongs to a Kafka topic and a specific partition.</p><p>Partition selection can depend on the record key and producer configuration.</p><p>When ordering is important for a particular entity, partitioning strategy becomes an architectural concern rather than merely an infrastructure setting.</p><p>For example, if all events associated with a given order are routed consistently to the same partition, their relative ordering can be maintained within that partition.</p><h3>4.3 Consumer Group</h3><p>The Analytics Service consumes the Kafka topic through a consumer group.</p><p>A consumer group provides a logical processing boundary.</p><p>Multiple instances can participate in the same group, allowing Kafka to distribute partitions among them.</p><p>This creates a horizontal-scaling model:</p><pre>One consumer<br>      |<br>      v<br>Assigned partitions<br></pre><pre>Multiple consumers<br>      |<br>      v<br>Partitions distributed among instances</pre><p>The scaling effect is therefore tied to partition count and assignment rather than simply to the number of application instances.</p><h3>4.4 Offset Management</h3><p>Offsets provide a durable reference to a consumer’s position in a partition.</p><p>This allows a consumer to resume from a previously committed position rather than requiring the event source to resend the entire stream.</p><p>Offset management also introduces an important correctness boundary.</p><p>The system must define when an event is considered processed relative to:</p><ul><li>offset advancement;</li><li>business processing;</li><li>external side effects;</li><li>application state.</li></ul><p>A consumer that advances its position before completing the business effect can create one class of failure.</p><p>A consumer that repeats processing after recovery can create another.</p><p>Therefore, offset management and application-level idempotency must be considered together.</p><h3>4.5 Replay-Oriented Processing</h3><p>Because Kafka retains event history according to topic configuration, consumers can be designed around replay-oriented workflows.</p><p>A new consumer group can process historical events independently from an existing consumer group.</p><p>Alternatively, consumer position can be reset to an earlier point when the operational model allows it.</p><p>Replay can support use cases such as:</p><ul><li>rebuilding derived state;</li><li>recovering analytics;</li><li>validating a new consumer implementation;</li><li>backfilling downstream systems;</li><li>correcting processing logic.</li></ul><p>However, replay is not free.</p><p>Reprocessing historical events requires:</p><ul><li>idempotent or controlled business effects;</li><li>sufficient processing capacity;</li><li>clear state-management semantics;</li><li>monitoring;</li><li>operational controls.</li></ul><h3>5. Architectural Analysis</h3><p>The architecture provides several properties that are valuable in distributed systems.</p><h3>5.1 Temporal Decoupling</h3><p>The producer does not need the Analytics Service to be synchronously available when the event is published.</p><p>This is different from guaranteeing that the consumer will process the event immediately.</p><p>Instead, the architecture separates publication from consumption.</p><h3>5.2 Horizontal Processing</h3><p>Partitions provide the basis for distributing event processing across consumers in the same group.</p><p>This creates a relationship between:</p><pre>Partition count<br>      ↓<br>Consumer parallelism<br>      ↓<br>Potential processing throughput</pre><p>The relationship is not linear in all workloads.</p><p>Processing cost, serialization, network bandwidth, downstream dependencies, and consumer configuration can become bottlenecks.</p><p>Therefore, partition count should be treated as an architectural decision rather than a simple performance-tuning parameter.</p><h3>5.3 Consumer Independence</h3><p>Different consumer groups can process the same topic independently.</p><p>For example:</p><pre>OrderCreated<br>                      Topic<br>                       |<br>          +------------+------------+<br>          |            |            |<br>          v            v            v<br>      Analytics      Audit      Recommendation<br>        Group         Group          Group</pre><p>Each group can maintain its own processing position.</p><p>This is a fundamental difference between event streaming and a single shared work queue.</p><p>Multiple logical applications can consume the same event history independently.</p><h3>5.4 Event Replay</h3><p>Replay provides a mechanism for reconstructing downstream state from historical events.</p><p>This can reduce dependence on one-time delivery for some analytical and derived-state use cases.</p><p>However, replay does not eliminate the need for correctness.</p><p>Consumers must understand whether processing an old event again is safe, whether state should be replaced or incrementally updated, and whether external side effects need protection.</p><h3>6. Resilience and Failure Semantics</h3><p>Resilience in Kafka-based systems must be analyzed at multiple layers.</p><h3>Producer Layer</h3><p>The producer must handle failures when publishing events.</p><h3>Broker Layer</h3><p>Kafka must maintain the event stream according to its configured durability and replication model.</p><h3>Consumer Layer</h3><p>Consumers must tolerate temporary unavailability and resume processing according to their group and offset state.</p><h3>Application Layer</h3><p>The business operation must remain correct when processing is repeated or resumed.</p><p>The laboratory README defines a Kafka consumer-outage scenario as planned future work: stop Analytics Service, publish orders, restart the service, and measure consumer lag, recovery time, and events processed after recovery.</p><p>Because that experiment has not been executed for this article, <strong>no recovery time, lag value, or throughput result is claimed here</strong>.</p><p>This distinction is methodologically important.</p><p>The architecture supports the scenario, but architectural capability is not equivalent to experimentally demonstrated performance.</p><p>The same principle applies to exactly-once claims.</p><p>Kafka provides delivery and processing mechanisms, but application correctness depends on how offsets, transactions, external side effects, and idempotency are implemented.</p><p>Therefore, a resilient Kafka consumer should be designed around explicit processing semantics rather than assuming that the messaging platform alone guarantees business-level exactly-once effects.</p><h3>7. Architectural Trade-Offs</h3><p>Kafka is not a universal replacement for queue-based messaging.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*q20d-L2Bc563tuzrDFkYOw.png" /></figure><p>SQS is well suited to workloads where the central abstraction is work that should be processed asynchronously.</p><p>SNS combined with SQS is useful when one event needs to reach multiple independent operational consumers.</p><p>Kafka becomes particularly relevant when event history, partition-based parallelism, independent consumer groups, and replay-oriented processing are important.</p><p>The choice should therefore be based on:</p><ul><li>communication semantics;</li><li>recovery requirements;</li><li>workload characteristics;</li><li>operational capabilities;</li><li>business meaning of the data.</li></ul><p>Using Kafka where a simple queue would be sufficient can introduce unnecessary complexity.</p><p>Conversely, using only queue semantics where multiple independent consumers need access to retained event history can constrain future architectural options.</p><h3>8. Broader Technological Relevance</h3><p>Event streaming is relevant to distributed applications because it provides a communication model in which producers and consumers can evolve with less direct temporal coupling.</p><p>The architectural concepts examined here — partitioned processing, consumer groups, offset management, and replay — apply to systems that must coordinate independent services while processing continuously generated event data.</p><p>Potential application domains include:</p><ul><li>financial transaction processing;</li><li>logistics and supply-chain systems;</li><li>e-commerce platforms;</li><li>telemetry and observability pipelines;</li><li>real-time analytics;</li><li>distributed data integration;</li><li>cloud-native application platforms.</li></ul><p>The relevance of these mechanisms does not depend on a specific industry.</p><p>The underlying problem is the same:</p><blockquote><strong><em>How can state changes propagate across distributed components while maintaining scalability, recoverability, and manageable operational complexity?</em></strong></blockquote><p>From an engineering perspective, the important contribution is not the use of Kafka as a product.</p><p>It is the architectural reasoning required to select event-streaming semantics when the system benefits from:</p><ul><li>retained event history;</li><li>independent consumers;</li><li>partition-based parallelism;</li><li>controlled processing position.</li></ul><h3>9. Reproducibility</h3><p>The accompanying <strong>Resilient Event-Driven Microservices Lab</strong> provides a concrete implementation context for the architecture described in this article.</p><p>The repository contains:</p><ul><li>Java/Spring Boot microservices;</li><li>Docker Compose infrastructure;</li><li>Kafka configuration;</li><li>LocalStack configuration;</li><li>architecture documentation;</li><li>experiment planning;</li><li>evidence from previously executed SNS/SQS experiments.</li></ul><p>The repository currently documents the Kafka path as:</p><pre>Order Service<br>      |<br>      v<br>    Kafka<br>      |<br>      v<br>Analytics Service</pre><p>and identifies Kafka event streaming, consumer groups, and replay-oriented processing as architectural patterns of the laboratory.</p><p>The local environment exposes Kafka on localhost:9092 and provides commands for starting the Analytics Service together with the other microservices.</p><p>This makes the article reproducible at the architectural level even though the Kafka performance and recovery experiments described as future work have not yet been executed.</p><h3>10. Limitations</h3><p>This article has an intentionally architectural scope.</p><p>No controlled Kafka performance benchmark was executed for this version.</p><p>Therefore, the article does not claim measured:</p><ul><li>throughput;</li><li>p50 latency;</li><li>p95 latency;</li><li>p99 latency;</li><li>consumer lag;</li><li>recovery time;</li><li>CPU utilization;</li><li>memory utilization.</li></ul><p>The local laboratory also does not represent a production multi-broker Kafka deployment.</p><p>Consequently, this work does not characterize:</p><ul><li>geographically distributed failures;</li><li>production network latency;</li><li>broker-failure behavior at scale;</li><li>cloud infrastructure costs;</li><li>production autoscaling behavior.</li></ul><p>The architecture should therefore be interpreted as a reference implementation and analytical model rather than as a production benchmark.</p><p>This limitation is deliberate.</p><p>Separating architectural analysis from measured performance prevents synthetic or unverified numbers from being presented as empirical evidence.</p><h3>11. Future Work</h3><p>The next stage of the laboratory can convert the architectural analysis into controlled experiments.</p><h3>11.1 Kafka Consumer Outage</h3><p>Stop Analytics Service while publishing events, then restore the consumer.</p><p>Measure:</p><ul><li>consumer lag;</li><li>recovery time;</li><li>number of events processed after recovery;</li><li>offset progression.</li></ul><h3>11.2 Consumer Group Scaling</h3><p>Run multiple Analytics Service instances in the same consumer group.</p><p>Compare processing behavior with different partition and consumer counts.</p><p>Measure:</p><ul><li>throughput;</li><li>processing latency;</li><li>consumer lag;</li><li>partition assignment.</li></ul><h3>11.3 Replay Experiment</h3><p>Publish a controlled set of events, process them, and then intentionally move consumer position backward or use a new consumer group.</p><p>Measure:</p><ul><li>number of historical events replayed;</li><li>processing time;</li><li>resulting state;</li><li>duplicate business effects.</li></ul><h3>11.4 Partitioning Strategy</h3><p>Evaluate different event-key strategies and determine how records are distributed across partitions.</p><p>The experiment should examine both load distribution and ordering requirements.</p><h3>11.5 Observability</h3><p>Add metrics and distributed tracing to correlate:</p><pre>eventId<br>   ↓<br>Producer publication<br>   ↓<br>Kafka partition<br>   ↓<br>Consumer group<br>   ↓<br>Offset<br>   ↓<br>Business processing</pre><p>This would provide a more complete view of event propagation across service boundaries.</p><h3>11.6 SQS versus Kafka</h3><p>The repository already identifies a future comparison between SQS and Kafka based on throughput, latency, consumer recovery, backlog, CPU, and memory.</p><p>That comparison should only be published after equivalent workloads have been executed and the resulting measurements have been recorded.</p><h3>12. Conclusion</h3><p>This article presented an architectural analysis of Apache Kafka within an event-driven microservice system implemented with Java and Spring Boot.</p><p>The reference architecture uses Kafka to separate Order Service event publication from Analytics Service consumption.</p><p>Kafka’s topic, partition, consumer-group, and offset abstractions provide a foundation for:</p><ul><li>independent event processing;</li><li>horizontal consumer scaling;</li><li>controlled processing position;</li><li>replay-oriented workflows.</li></ul><p>The analysis also demonstrates why these mechanisms must be considered as architectural semantics rather than simply as messaging features.</p><p>Partition count affects consumer parallelism.</p><p>Consumer groups create independent processing boundaries.</p><p>Offsets determine consumer position.</p><p>Replay changes the point in the event history from which processing occurs.</p><p>At the same time, these capabilities introduce additional design responsibilities.</p><p>Consumers must define:</p><ul><li>processing semantics;</li><li>idempotency requirements;</li><li>state reconstruction strategies;</li><li>operational monitoring;</li><li>recovery behavior.</li></ul><p>No performance or recovery benchmark was claimed because those experiments have not yet been executed.</p><p>Instead, the article establishes a technically grounded reference architecture and a reproducible path toward future experimentation.</p><p>The broader conclusion is that event streaming becomes valuable when a distributed system requires more than asynchronous message delivery.</p><p>It becomes particularly relevant when the system requires:</p><blockquote><strong><em>retained event history, independent consumers, partition-based parallelism, and controlled processing position.</em></strong></blockquote><p>Kafka should therefore be selected not because it is a popular messaging technology, but because its event-streaming semantics match the communication and processing requirements of the system.</p><h3>References</h3><p><strong>[1]</strong> Jorge Matheus, <em>Resilient Event-Driven Microservices Lab</em>, GitHub repository, 2026.</p><p><strong>[2]</strong> Apache Software Foundation, <em>Basic Kafka Operations</em>, Apache Kafka Documentation.</p><p><strong>[3]</strong> Apache Software Foundation, <em>Consumer Configs</em>, Apache Kafka Documentation.</p><p><strong>[4]</strong> Apache Software Foundation, <em>Application Reset Tool</em>, Apache Kafka Documentation.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b0c4fdc676e5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Designing Resilient Event-Driven Microservices on AWS]]></title>
            <link>https://jorgenunes10.medium.com/designing-resilient-event-driven-microservices-on-aws-fd24902a3273?source=rss-6f61c962a302------2</link>
            <guid isPermaLink="false">https://medium.com/p/fd24902a3273</guid>
            <category><![CDATA[software-architecture]]></category>
            <category><![CDATA[apache-kafka]]></category>
            <category><![CDATA[microservices]]></category>
            <category><![CDATA[distributed-systems]]></category>
            <category><![CDATA[aws]]></category>
            <dc:creator><![CDATA[Jorge Nunes]]></dc:creator>
            <pubDate>Wed, 30 Sep 2026 17:52:53 GMT</pubDate>
            <atom:updated>2026-09-30T19:01:43.365Z</atom:updated>
            <content:encoded><![CDATA[<h3>A Java-Based Architecture Using Amazon SQS, Amazon SNS, and Apache Kafka</h3><h3>Abstract</h3><p>Modern distributed applications increasingly rely on asynchronous communication to reduce temporal coupling, isolate failures, and support independent scaling of service components. However, introducing multiple messaging mechanisms also creates architectural challenges related to delivery semantics, duplicate processing, backlog management, and operational complexity.</p><p>This article presents a Java-based reference architecture for resilient event-driven microservices that combines Amazon Simple Queue Service (SQS), Amazon Simple Notification Service (SNS), and Apache Kafka. The architecture models an order-processing workflow composed of independent Order, Payment, Inventory, and Analytics services. Amazon SNS and SQS provide event fan-out and asynchronous work distribution, while Apache Kafka provides event-streaming capabilities for analytics-oriented processing.</p><p>A reproducible local laboratory was implemented using Java, Spring Boot, Docker Compose, Kafka, and LocalStack. Two controlled experiments were conducted. The first evaluated consumer outage and queue backlog recovery. The second evaluated message redelivery when processing exceeded the configured SQS visibility timeout. The experiments demonstrated that asynchronous queues can buffer work during temporary consumer outages and that the same business event can be delivered multiple times when processing does not complete before the visibility timeout expires.</p><p>The results reinforce an important architectural principle: messaging infrastructure can provide temporal decoupling and failure isolation, but application-level mechanisms such as idempotency are still required to prevent duplicate deliveries from producing duplicate business effects.</p><p><strong>Keywords:</strong> distributed systems, microservices, event-driven architecture, AWS, Amazon SQS, Amazon SNS, Apache Kafka, resilience, fault isolation, idempotency.</p><h3>1. Introduction</h3><p>Microservice architectures decompose applications into independently deployable services that communicate through well-defined interfaces. This decomposition can improve organizational and technical autonomy, but it also introduces distributed-systems concerns that are less prominent in monolithic applications. Network failures, service outages, retries, message duplication, inconsistent state, and partial failures become normal operating conditions rather than exceptional events.</p><p>Microservice architecture has evolved from service-oriented computing and emphasizes independently deployable components with explicit communication boundaries. Research on microservices identifies both architectural benefits and challenges associated with service decomposition, distributed communication, deployment, and operational complexity.</p><p>Synchronous HTTP communication is appropriate for many interactions, but it creates temporal coupling between the caller and the downstream service. If the downstream service becomes unavailable, the caller may experience increased latency, errors, or cascading failures.</p><p>Asynchronous messaging provides an alternative. A producer can publish work without requiring the downstream consumer to be available at the same instant. Amazon SQS is explicitly designed to decouple sending and receiving components, allowing messages to remain in a queue while consumers are temporarily unavailable.</p><p>However, asynchronous architectures do not eliminate failure. They change its form.</p><p>A service outage can produce queue backlog. Retry mechanisms can produce repeated delivery. A consumer can process the same business event more than once. Streaming systems introduce additional concerns around offsets, partitions, consumer groups, and replay.</p><p>This article investigates the following research question:</p><blockquote><strong><em>How can heterogeneous messaging mechanisms be combined within an event-driven microservice architecture to improve decoupling, scalability, and failure isolation while maintaining manageable operational complexity?</em></strong></blockquote><p>To investigate this question, a reference architecture was implemented using Java, Spring Boot, Amazon SNS, Amazon SQS, and Apache Kafka. The implementation was accompanied by controlled failure experiments designed to demonstrate asynchronous recovery and message redelivery.</p><h3>2. Technical Background</h3><h3>2.1 Event-Driven Architecture</h3><p>In an event-driven architecture, components communicate through events representing facts that have occurred in the system.</p><p>For example:</p><pre>OrderCreated<br>PaymentAuthorized<br>InventoryReserved<br>OrderCompleted</pre><p>Rather than directly invoking another service, a producer can publish an event and allow interested consumers to process it independently.</p><p>This approach reduces direct temporal coupling between services, but introduces eventual consistency and distributed coordination concerns.</p><p>The architecture therefore shifts part of the system’s complexity from synchronous request/response interactions toward messaging semantics and asynchronous state management.</p><h3>2.2 Amazon SQS</h3><p>Amazon Simple Queue Service provides queue-based asynchronous communication.</p><p>A producer places a message into a queue, while one or more consumers retrieve messages for processing.</p><p>The queue acts as a temporal buffer:</p><pre>Producer<br>   |<br>   v<br>+-------+<br>|  SQS  |<br>+-------+<br>   |<br>   v<br>Consumer</pre><p>This model is useful when producers and consumers operate at different rates or when consumers may temporarily become unavailable.</p><p>An important SQS mechanism is the <strong>visibility timeout</strong>. After a consumer receives a message, the message becomes temporarily invisible to other consumers. If the consumer does not successfully delete the message before the visibility period expires, the message can become available again. AWS documents visibility timeout as a mechanism that helps prevent other consumers from processing a message while the original processing attempt is still active.</p><p>This behavior makes visibility-timeout configuration an important part of consumer design.</p><h3>2.3 Amazon SNS and SQS Fan-Out</h3><p>Amazon SNS provides a publish/subscribe model in which a publisher sends an event to a topic and the notification can be delivered to multiple subscribers.</p><p>SNS can be combined with independent SQS queues:</p><pre>                    SNS Topic<br>                       |<br>             +---------+---------+<br>             |                   |<br>             v                   v<br>       Payment Queue       Inventory Queue<br>             |                   |<br>             v                   v<br>      Payment Service      Inventory Service</pre><p>AWS documents this SNS-to-SQS pattern as a mechanism for asynchronous fan-out in which messages can persist in independent SQS queues for subsequent processing.</p><p>This separation means that Payment Service and Inventory Service do not need to process the event at exactly the same time or at the same rate.</p><h3>2.4 Apache Kafka</h3><p>Apache Kafka provides a distributed event-streaming model based on topics and partitions.</p><p>Kafka consumers can be organized into consumer groups. Within a consumer group, partitions are distributed among consumer instances, allowing processing to scale across multiple consumers. Kafka also maintains offsets that allow consumers to track their position in the event stream.</p><p>This makes Kafka particularly useful for use cases involving:</p><ul><li>event streams;</li><li>analytics;</li><li>independent consumer groups;</li><li>replay-oriented processing;</li><li>partition-based parallelism.</li></ul><p>The architecture in this work therefore uses Kafka for Analytics Service consumption while using SNS and SQS for operational service communication.</p><h3>3. Proposed Architecture</h3><p>The reference architecture contains four primary microservices.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*us0-454TxzzttcMJzgWHMQ.png" /><figcaption>Figure 1. Reference architecture combining Amazon SNS, Amazon SQS, and Apache Kafka for asynchronous communication between microservices.</figcaption></figure><p>The implementation uses the following communication patterns:</p><ol><li>SQS for asynchronous work buffering and independent consumer processing.</li><li><strong>SNS → SQS</strong> for event fan-out to independent consumers.</li><li><strong>Kafka</strong> for event streaming and analytics processing.</li></ol><p>The accompanying GitHub repository contains the complete laboratory implementation and infrastructure configuration.</p><p><strong>Source repository:</strong><br><a href="https://github.com/jorgematheus/resilient-event-driven-microservices-lab">Resilient Event-Driven Microservices Lab — GitHub</a></p><h3>4. Communication Model</h3><p>When a client creates an order, the Order Service generates an OrderCreated event.</p><p>The event contains information such as:</p><pre>{<br>  &quot;eventId&quot;: &quot;f7e953f7-f4fd-4c86-b182-1b1628e7b3bd&quot;,<br>  &quot;type&quot;: &quot;OrderCreated&quot;,<br>  &quot;orderId&quot;: &quot;c0b471df-47f2-4b12-8e2e-8dc78d33ef93&quot;,<br>  &quot;customerId&quot;: &quot;experimental-duplicate&quot;,<br>  &quot;amount&quot;: 149.90<br>}</pre><p>The Order Service publishes the event through two asynchronous paths.</p><h3>Operational processing</h3><pre>Order Service<br>      |<br>      v<br>   SNS Topic<br>      |<br>      +------------+<br>      |            |<br>      v            v<br> Payment Queue   Inventory Queue<br>      |            |<br>      v            v<br> Payment         Inventory<br> Service         Service</pre><h3>Analytics processing</h3><pre>Order Service<br>      |<br>      v<br> Kafka Topic<br>      |<br>      v<br>Analytics Service</pre><p>This design separates operational service processing from streaming-oriented analytics.</p><h3>5. Resilience Mechanisms</h3><h3>5.1 Asynchronous Buffering</h3><p>SQS provides a buffer between producers and consumers.</p><p>If Payment Service becomes unavailable, messages can remain in the queue until the consumer becomes available again.</p><p>This reduces direct temporal coupling between Order Service and Payment Service.</p><h3>5.2 Retry and Redelivery</h3><p>A failed or incomplete processing attempt does not necessarily mean that a message has been permanently lost.</p><p>When a message becomes visible again, the consumer may receive it again.</p><p>This requires consumers to distinguish between:</p><pre>message delivery</pre><p>and:</p><pre>business operation</pre><p>The two concepts are not equivalent.</p><h3>5.3 Idempotency</h3><p>If a business event can be delivered multiple times, the consumer must prevent repeated deliveries from producing repeated business effects.</p><p>A stable identifier such as:</p><pre>eventId</pre><p>can be used as an idempotency key.</p><p>Conceptually:</p><pre>Event received<br>      |<br>      v<br>Already processed?<br>   /          \<br> yes           no<br> |              |<br>Ignore        Process<br>                 |<br>                 v<br>        Record eventId</pre><p>The objective is not to guarantee that the messaging infrastructure never delivers an event twice.</p><p>The objective is to make repeated delivery safe.</p><h3>5.4 Dead-Letter Queues</h3><p>Repeated processing failures can eventually be routed to a dead-letter queue.</p><p>A dead-letter strategy prevents a permanently problematic message from continuously consuming consumer capacity.</p><p>In a production implementation, retry limits, backoff policies, and dead-letter handling should be explicitly designed according to the business semantics of the event.</p><h3>6. Experimental Methodology</h3><p>The architecture was implemented as a reproducible local laboratory using:</p><ul><li>Java 21;</li><li>Spring Boot;</li><li>Docker Compose;</li><li>Apache Kafka;</li><li>LocalStack;</li><li>Amazon SNS and SQS-compatible APIs.</li></ul><p>The repository provides the source code, infrastructure configuration, architecture documentation, and experiment documentation.</p><p>The experiments were designed to evaluate behavioral properties rather than production-scale performance.</p><p>Two experiments were executed.</p><h3>Experiment 1 — Consumer Outage and Queue Backlog Recovery</h3><p>The Payment Service consumer was temporarily unavailable while messages accumulated in payment-queue.</p><p>After the consumer was restored, the queue backlog was observed as the consumer progressively processed the accumulated messages.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Z_hzM_pP9qw-lmGXBVApGQ.png" /><figcaption>Queue backlog during temporary Payment Service consumer unavailability.</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DWykUGEVihHfGcPZUpg5fw.png" /><figcaption>Queue returning to zero after consumer recovery.</figcaption></figure><h3>Experiment 2 — SQS Message Redelivery</h3><p>The Payment Service consumer was configured with:</p><pre>Visibility timeout: 5 seconds<br>Processing duration: 15 seconds</pre><p>The processing duration was intentionally longer than the visibility timeout.</p><p>This created a controlled condition in which the same message could become visible again before the original processing attempt completed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2uQ6rUUlahFo-S9dX8jd0g.png" /></figure><h3>7. Experimental Results</h3><h3>7.1 Consumer Outage and Queue Backlog Recovery</h3><p>The first experiment demonstrated queue buffering during temporary consumer unavailability.</p><p>The observed queue depth progressed approximately as follows:</p><p>ObservationQueue depthInitial backlog~100Recovery in progress~80Recovery in progress~20Recovery completed0</p><p>The sequence demonstrates that messages remained available in the queue while the Payment Service consumer was unavailable.</p><p>After the consumer was restored, the backlog progressively decreased until the queue reached zero.</p><p>The result can be represented as:</p><pre>Consumer unavailable<br>        |<br>        v<br>Queue backlog increases<br>        |<br>        v<br>Consumer restored<br>        |<br>        v<br>Backlog decreases<br>        |<br>        v<br>Queue = 0</pre><p>This experiment demonstrates the buffering and recovery characteristics of asynchronous queue-based communication.</p><p>The result should not be interpreted as a production throughput benchmark. The workload and environment were intentionally small and controlled.</p><h3>7.2 SQS Message Redelivery</h3><p>The second experiment produced a direct observation of repeated delivery.</p><p>The consumer was configured with a five-second visibility timeout and a 15-second processing delay.</p><p>The same SQS message was received three times:</p><pre>RECEIVED messageId=3e5920d9-40d0-4442-9c1c-e8726c10215a</pre><pre>RECEIVED messageId=3e5920d9-40d0-4442-9c1c-e8726c10215a</pre><pre>RECEIVED messageId=3e5920d9-40d0-4442-9c1c-e8726c10215a</pre><p>The application event identifier remained the same:</p><pre>eventId=f7e953f7-f4fd-4c86-b182-1b1628e7b3bd</pre><p>The experiment therefore observed:</p><pre>One business event<br>        |<br>        +--&gt; Delivery 1<br>        |<br>        +--&gt; Delivery 2<br>        |<br>        +--&gt; Delivery 3</pre><p>The observation provides direct experimental evidence that message delivery and business-event uniqueness are separate concerns.</p><p>Because processing intentionally exceeded the visibility timeout, the same message became eligible for subsequent delivery before the original processing attempt completed.</p><h3>8. Discussion</h3><p>The two experiments demonstrate complementary resilience properties.</p><p>The first experiment demonstrated <strong>buffering and recovery</strong>.</p><p>When the Payment Service was unavailable, messages accumulated in SQS rather than requiring the producer to wait synchronously for the downstream service.</p><p>The second experiment demonstrated <strong>redelivery and duplicate-processing risk</strong>.</p><p>The same event was observed multiple times when processing exceeded the configured visibility timeout.</p><p>These observations lead to an important distinction:</p><blockquote><strong><em>Asynchronous buffering does not imply exactly-once business processing.</em></strong></blockquote><p>A messaging infrastructure can provide temporal decoupling while the application still needs mechanisms to handle repeated delivery safely.</p><p>This distinction is particularly important for operations such as:</p><ul><li>charging a customer;</li><li>reserving inventory;</li><li>creating financial records;</li><li>sending external notifications;</li><li>triggering downstream workflows.</li></ul><p>For these operations, repeated delivery can produce an incorrect business result unless the consumer is designed for idempotent processing.</p><h3>9. Idempotency Implications</h3><p>The duplicate-delivery experiment provides the foundation for the next stage of the laboratory.</p><p>A Payment Service can use the application-level eventId as an idempotency key.</p><p>For example:</p><pre>eventId<br>------------------------------------<br>f7e953f7-f4fd-4c86-b182-1b1628e7b3bd</pre><p>Before applying a business operation, the service can verify whether this identifier has already been processed.</p><p>The expected behavior becomes:</p><pre>Delivery 1<br>    |<br>    v<br>eventId not found<br>    |<br>    v<br>Process business operation<br>    |<br>    v<br>Record eventId<br></pre><pre>Delivery 2<br>    |<br>    v<br>eventId already exists<br>    |<br>    v<br>Skip business operation</pre><p>The messaging system may therefore deliver the event multiple times while the application still produces only one business effect.</p><p>This is an important resilience boundary: <strong>delivery semantics belong to the messaging layer, while business idempotency belongs to the application layer.</strong></p><h3>10. Architectural Trade-Offs</h3><p>The architecture deliberately combines different messaging mechanisms rather than attempting to use one technology for every communication requirement.</p><p>MechanismPrimary roleArchitectural characteristicSQSPoint-to-point workQueue-based asynchronous processingSNS → SQSEvent fan-outIndependent consumer queuesKafkaEvent streamingPartitioned streams and consumer groups</p><p>SQS is appropriate when a service needs to consume work independently from the producer.</p><p>SNS combined with SQS is useful when the same event needs to reach multiple independent consumers without requiring a single shared queue.</p><p>Kafka is useful when event streams, partitions, consumer groups, and replay-oriented processing are important.</p><p>These mechanisms are complementary rather than interchangeable. The architectural decision should therefore be driven by communication semantics and operational requirements rather than by technology preference alone.</p><h3>11. Broader Technological Relevance</h3><p>The experiments illustrate a broader principle in distributed-system engineering: resilience is not a single infrastructure feature.</p><p>It emerges from the interaction between:</p><pre>Messaging infrastructure<br>        +<br>Failure semantics<br>        +<br>Consumer behavior<br>        +<br>State management<br>        +<br>Operational recovery</pre><p>Asynchronous queues can reduce temporal coupling and provide buffering during temporary failures. At the same time, they introduce operational concerns such as:</p><ul><li>backlog growth;</li><li>repeated delivery;</li><li>retry behavior;</li><li>duplicate processing;</li><li>eventual consistency;</li><li>dead-letter handling;</li><li>consumer recovery.</li></ul><p>The architecture therefore treats messaging systems as architectural components with explicit failure semantics rather than merely as transport mechanisms.</p><p>This perspective is relevant to distributed applications where independent services must continue operating despite partial failures.</p><h3>12. Reproducibility</h3><p>The complete implementation accompanying this article is publicly available in the <strong>Resilient Event-Driven Microservices Lab</strong> repository.</p><p><a href="https://github.com/jorgematheus/resilient-event-driven-microservices-lab">GitHub — Resilient Event-Driven Microservices Lab</a></p><p>The repository contains:</p><ul><li>Java/Spring Boot microservices;</li><li>Docker Compose configuration;</li><li>LocalStack initialization;</li><li>Kafka configuration;</li><li>architecture documentation;</li><li>experimental plans;</li><li>consumer-outage evidence;</li><li>duplicate-delivery evidence.</li></ul><p>The repository currently documents the following communication patterns:</p><ol><li>SQS point-to-point work distribution.</li><li>SNS → SQS event fan-out.</li><li>Kafka event streaming.</li></ol><p>The first two resilience experiments have been executed and documented:</p><ul><li>Consumer Outage and Queue Backlog Recovery.</li><li>SQS Message Redelivery.</li></ul><p>The remaining experiments are maintained as planned extensions of the laboratory.</p><p>The use of a reproducible laboratory makes it possible to inspect the implementation and repeat the scenarios independently.</p><h3>13. Limitations</h3><p>The experiments were conducted in a local environment using Docker Compose and LocalStack.</p><p>Therefore, the results should not be interpreted as production performance benchmarks.</p><p>The experiments primarily evaluate behavioral properties:</p><ul><li>asynchronous buffering;</li><li>consumer recovery;</li><li>backlog drainage;</li><li>message redelivery;</li><li>duplicate delivery of the same business event.</li></ul><p>They do not yet evaluate:</p><ul><li>large-scale throughput;</li><li>multi-node production failures;</li><li>regional failures;</li><li>network partitions across geographically distributed environments;</li><li>production AWS latency;</li><li>infrastructure autoscaling;</li><li>production observability overhead.</li></ul><p>The duplicate-delivery experiment also intentionally uses an artificially short five-second visibility timeout and a 15-second processing delay. These values were selected to make the failure condition reproducible and observable.</p><h3>14. Future Work</h3><p>The laboratory provides several directions for further experimentation.</p><h3>14.1 Idempotent Consumer</h3><p>Implement persistent event tracking and repeat the duplicate-delivery experiment.</p><p>The objective would be to compare:</p><pre>3 deliveries<br>        ↓<br>3 business effects</pre><p>with:</p><pre>3 deliveries<br>        ↓<br>1 business effect</pre><p>This would provide experimental evidence for the effectiveness of idempotency.</p><h3>14.2 Kafka Consumer Recovery</h3><p>Stop Analytics Service while publishing events and measure:</p><ul><li>consumer lag;</li><li>recovery time;</li><li>number of events processed after recovery;</li><li>offset progression.</li></ul><h3>14.3 Throughput Comparison</h3><p>Generate a controlled workload and compare:</p><ul><li>throughput;</li><li>p50 latency;</li><li>p95 latency;</li><li>p99 latency;</li><li>backlog;</li><li>CPU;</li><li>memory.</li></ul><p>The comparison should be performed under equivalent workload conditions.</p><h3>14.4 Retry and Dead-Letter Behavior</h3><p>Introduce controlled processing failures and evaluate:</p><ul><li>retry frequency;</li><li>retry delay;</li><li>maximum attempts;</li><li>dead-letter routing;</li><li>recovery after transient failures.</li></ul><h3>14.5 Observability</h3><p>Extend the laboratory with distributed tracing and metrics to correlate:</p><pre>eventId<br>    ↓<br>SNS<br>    ↓<br>SQS<br>    ↓<br>Consumer<br>    ↓<br>Business operation</pre><p>This would make it possible to analyze message propagation and failure recovery across service boundaries.</p><h3>15. Conclusion</h3><p>This article presented a Java-based event-driven microservice architecture combining Amazon SNS, Amazon SQS, and Apache Kafka.</p><p>The architecture separates operational service communication from event-streaming workloads and demonstrates how heterogeneous messaging mechanisms can coexist within a distributed system.</p><p>Two controlled experiments were executed.</p><p>The first demonstrated that a temporary Payment Service consumer outage can cause messages to accumulate in an SQS queue and that the accumulated backlog can subsequently be processed after consumer recovery.</p><p>The second demonstrated that the same business event can be delivered multiple times when message processing exceeds the configured visibility timeout.</p><p>Together, these experiments illustrate an important property of resilient distributed systems:</p><blockquote><strong><em>Failure isolation and asynchronous recovery require both infrastructure-level mechanisms and application-level correctness guarantees.</em></strong></blockquote><p>Queues can buffer work. Publishers can remain decoupled from temporary consumer failures. Streaming systems can provide independent event consumption. However, consumers must still account for repeated delivery and partial failure.</p><p>The resulting architecture therefore treats resilience as a cross-layer property involving messaging semantics, service behavior, state management, and recovery strategies.</p><p>The accompanying GitHub laboratory provides the implementation and experimental evidence required to reproduce the scenarios and serves as a foundation for subsequent experiments involving idempotency, Kafka recovery, retry behavior, dead-letter queues, performance characterization, and observability.</p><h3>References</h3><p>[1] N. Dragoni et al., “Microservices: Yesterday, Today, and Tomorrow,” in <em>Present and Ulterior Software Engineering</em>, Springer, 2017, pp. 195–216. DOI: 10.1007/978–3–319–67425–4_12.</p><p>[2] Amazon Web Services, “Configuring visibility timeouts in Amazon SQS,” <em>Amazon Simple Queue Service Developer Guide</em>.</p><p>[3] Amazon Web Services, “Fanout Amazon SNS notifications to Amazon SQS queues for asynchronous processing,” <em>Amazon Simple Notification Service Developer Guide</em>.</p><p>[4] Amazon Web Services, “What is Amazon SNS?” <em>Amazon Simple Notification Service Developer Guide</em>.</p><p>[5] Apache Software Foundation, “Introduction — Consumers and Consumer Groups,” <em>Apache Kafka Documentation</em>.</p><p>[6] Apache Software Foundation, “Basic Kafka Operations — Managing Consumer Groups,” <em>Apache Kafka Documentation</em>.</p><p>[7] Jorge Matheus, “Resilient Event-Driven Microservices Lab,” GitHub repository, 2026. <a href="https://github.com/jorgematheus/resilient-event-driven-microservices-lab">https://github.com/jorgematheus/resilient-event-driven-microservices-lab</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fd24902a3273" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>