<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Hamzaazam on Medium]]></title>
        <description><![CDATA[Stories by Hamzaazam on Medium]]></description>
        <link>https://medium.com/@hamzaazam.web?source=rss-3cdc1c343c6b------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*4RS6FHv_FqixywLnRa3fqA.jpeg</url>
            <title>Stories by Hamzaazam on Medium</title>
            <link>https://medium.com/@hamzaazam.web?source=rss-3cdc1c343c6b------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 22:34:31 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@hamzaazam.web/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[How I Designed a Secure ECS Architecture with Multi-Environment Routing Through a Private Load…]]></title>
            <link>https://medium.com/@hamzaazam.web/how-i-designed-a-secure-ecs-architecture-with-multi-environment-routing-through-a-private-load-c79daf03874f?source=rss-3cdc1c343c6b------2</link>
            <guid isPermaLink="false">https://medium.com/p/c79daf03874f</guid>
            <dc:creator><![CDATA[Hamzaazam]]></dc:creator>
            <pubDate>Sat, 10 Jan 2026 06:57:01 GMT</pubDate>
            <atom:updated>2026-01-10T06:57:01.613Z</atom:updated>
            <content:encoded><![CDATA[<h3>How I Designed a Secure ECS Architecture with Multi-Environment Routing Through a Private Load Balancer</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/818/1*v6t2G6mi8YCMugKQY5TdZw.png" /></figure><p>Modern apps don’t usually run in just one environment. They run in dev, staging, and production, each with its own needs for reliability, permissions, and traffic.</p><p>It wasn’t just deploying a new internal service on AWS ECS that was hard; it was also coming up with a secure, multi-environment routing architecture that didn’t add too much complexity.</p><p>Most guides show you how to use ECS, but not <strong>why the architecture</strong> is the way it is.</p><p>This article explains the decisions I made and why I made it, so you can think like an architect instead of just following directions.</p><h4><strong>1. The Requirements That Made the Architecture</strong></h4><p>I broke the problem down into four simple rules before I started building anything.</p><p><strong>A) Multiple environments</strong></p><ul><li>Dev should work on its own</li><li>Staging should act like a preview for production.</li><li>Production should be completely separate.</li></ul><p>B) <strong>Traffic must stay private</strong></p><ul><li>No endpoints that are open to the public.</li><li>No accidental exposure to the internet.</li><li>The service needed to be limited to the VPC.</li></ul><p><strong>C) Don’t duplicate infrastructure.</strong></p><ul><li>I didn’t want 3 ALBs</li><li>Three sets of rules for networking</li><li>Three stacks for monitoring</li><li>Three times as much money</li><li>I needed a clear separation that wasn’t too complicated.</li></ul><p><strong>D) Routing that can grow and last for a long time</strong></p><p>If tomorrow a new setting (QA, Pilot) a new version (v2) or a different microservice needs to be added, it should fit in with the current system without any problems.</p><p>These rules helped in making every decision that came after.</p><h4>2. Why I Chose ECS and ECR for Deployment</h4><p>Setting this up was simple: ECR holds the images of the containers.</p><p>ECS (Fargate) runs the containers without EC2.</p><p>ECS Services take care of scaling and keeping the containers running.</p><p>Task Definitions tell you what the CPU, memory, image, environment variables, IAM role, and ports are.</p><p>The real problem wasn’t ECS itself; it was figuring out how to set up a shared, secure routing system for dev, staging, and prod.That’s when AWS networking really starts to become an architectural problem.</p><h4>3. Why the First Big Choice Was a Private Load Balancer</h4><p>There are two kinds of ALBs that AWS offers:</p><p>Public ALB: This type of ALB has a public IP address and can be reached from the internet.</p><p>Private ALB: stays in your subnets, doesn’t have a public IP, and can only be accessed from within your VPC.</p><p>Using a public ALB would have made my service available to the public, which was not what I wanted.</p><p>A private ALB keeps everything inside the VPC, makes security rules easier to follow, protects environments, and makes it harder for attackers to get in.</p><h4>4. The Main Idea: One private ALB and many target groups</h4><p>I used: instead of making three separate ALBs for dev, staging, and prod.</p><p>One private ALB</p><p>Three target groups, one for each environment.</p><p>Each target group linked to its own ECS service.</p><p>Routing was easy:</p><p>/Dev/* → Dev target group</p><p>/staging/* → Staging target group</p><p>/prod/* → Prod target group</p><p>This setup made it easier to separate environments, scale up, keep an eye on things, route traffic, and save money.</p><p>An ALB doesn’t care what’s running behind it; it just follows routing rules, so it works.</p><p>You can use target groups to separate environments with simple path-based rules.</p><h4>5. A full overview of the architecture</h4><p>This is how it goes from development to running service:</p><p><strong>Step 1: Send the image to ECR</strong></p><p>A CI pipeline builds the Docker image and sends it to ECR with tags like “dev,” “staging,” or “prod.”</p><p><strong>Step 2: Each environment has its own task definition.</strong></p><p>Each environment (dev, staging, prod) has its own task definition that includes CPU and memory. Variables in the environment, IAM role, Ports,Image of a container from ECR</p><p><strong>Step 3: Three ECS Clusters That Are Not Connected</strong></p><p>I used: instead of one cluster.</p><p>ECS Cluster — Dev</p><p>ECS Cluster — Staging</p><p>ECS Cluster — Prod</p><p>Each cluster has its own ECS service that: Uses the right task definition</p><p>Works inside private subnets.</p><p>Registers with its own group of targets.</p><p><strong>Step 4: Internal traffic goes to the private ALB.</strong></p><p>There is only one private ALB in the VPC, and it doesn’t have a public IP.</p><p>It listens on ports 80 and 443.</p><p><strong>Step 5: ALB Uses Paths to Direct Traffic</strong></p><p>All environments can use simple routing rules:</p><p>/dev/* → Target group for developers</p><p>/staging/* → Staging target group</p><p>/prod/* → Prod target group</p><p>Each target group points to tasks in its own ECS cluster.</p><p><strong>Step 6: Environments that are cleanly separated</strong></p><p>Dev traffic → Dev cluster</p><p>Traffic for staging → Staging cluster</p><p>Traffic to Prod → Cluster to Prod</p><p>No mixing, no overlap, and no chance that one environment will touch another.</p><h4>6. Security Design Choices (The Reason for It All)</h4><p><strong>Private subnets:</strong> All tasks take place in private subnets, not on the public internet.</p><p><strong>Tight security groups:</strong> The ALB only lets traffic from inside, and ECS tasks only let traffic from the ALB.</p><p>This keeps things from moving around inside the VPC that you don’t want them to.</p><p><strong>Different IAM roles:</strong> Prod tasks have stricter permissions than dev or staging tasks.</p><p><strong>No public endpoints:</strong>Unless a front-end or API Gateway needs to be public, everything stays private.</p><p><strong>Separate health checks:</strong> Each environment has its own health check path (like /dev/health, /staging/health, or /prod/health) to keep problems from spreading.</p><h4>7. This architecture gives me operational benefits</h4><p>✔ Scaling horizontally at the environment level. Dev may only need one task.Prod might need ten.No problems.</p><p>✔ Debugging is easier.It’s natural for logs and metrics to be separate.</p><p>✔ Cheaper. One ALB instead of three.</p><p>✔ Clear traffic directions.If someone hits /staging, it always goes to the staging containers.</p><p>✔ Design that will last.Need a QA space?, Just put in:</p><p>a qa-target-group</p><p>a qa-service</p><p>a new rule in the ALB</p><h4>8. What We Learnt</h4><p>1. <strong>Plan the architecture first.</strong></p><p>When you look at ECS screen by screen instead of as a whole system, it can be hard to understand.</p><p>2. <strong>Routing is an important part of the design, not just an extra.</strong></p><p>The way you set up your routing can affect security, scaling, isolation, and even cost.</p><p>3. <strong>Keep things that need to be separate and share things that make sense.</strong></p><p>Each environment should have its own ECS cluster, task definitions, IAM roles, and rules for scaling.</p><p>But they all use the same infrastructure, such as the VPC, subnets, and private ALB.</p><p>4. <strong>The choices you make early on are important later.</strong></p><p>Long-term, it will be worth it to choose a private ALB, clean path-based routing, and strong environment isolation.</p><p>These decisions affect the whole architecture, not just the deployment.</p><h4>9. Last Thoughts</h4><p>This deployment wasn’t just about putting containers on ECS.</p><p>It was about making a routing layer that was safe, could grow with the team, and knew about the environment.</p><p><strong>What did I learn the most?</strong></p><p>The value of cloud expertise isn’t in building infrastructure; it’s in making systems that are easy to understand, safe, scalable and maintainable.</p><p>If you’re using ECS in more than one environment, you might want to think about using one internal ALB, many target groups, isolated task definitions, strict groups for security.</p><p>It’s safer, cleaner, and durable.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c79daf03874f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Demystifying Message Queues: Seamless Communication for Asynchronous Processing]]></title>
            <link>https://medium.com/red-buffer/demystifying-message-queues-seamless-communication-for-asynchronous-processing-372ebcc9865b?source=rss-3cdc1c343c6b------2</link>
            <guid isPermaLink="false">https://medium.com/p/372ebcc9865b</guid>
            <category><![CDATA[software-architecture]]></category>
            <category><![CDATA[distributed-systems]]></category>
            <category><![CDATA[asynchronouscommunication]]></category>
            <category><![CDATA[message-queue]]></category>
            <category><![CDATA[scalability]]></category>
            <dc:creator><![CDATA[Hamzaazam]]></dc:creator>
            <pubDate>Tue, 12 Sep 2023 06:37:12 GMT</pubDate>
            <atom:updated>2023-09-12T06:37:12.657Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/640/1*5SEn3d3ELUU-Bg_wE-hLrA.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@polarmermaid?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Anne Nygård</a> on <a href="https://unsplash.com/photos/Sl69Jw-o0rU?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><p>In the ever-evolving world of software development, developers have faced a common challenge when building complex applications: efficient and seamless communication between different components and services. As the demand for scalability, reliability, and real-time processing increased, a solution was needed to overcome the limitations of traditional request-response models.</p><p>In the bustling realm of distributed systems, where multiple components work together to deliver a cohesive application, a new hero emerged: message queues. To shed light on their significance, let’s dive into the concept and explore why message queues have become an integral part of modern application architectures.</p><p>Imagine you’re developing an online food delivery application, where the order service is responsible for processing incoming requests. However, as the popularity of your app grows, the order service starts to struggle under the weight of numerous requests. Some orders get lost, others are processed out of turn, and worst of all, there’s no efficient way to update other system components about the status of orders.</p><p>Enter message queues, the saviors of communication challenges in distributed systems. By leveraging message queues, developers can maintain the sequence of orders by inserting them into a queue, ensuring they are processed in the correct order. Additionally, they can publish messages to notify other system components about the status of orders, enabling seamless collaboration and real-time updates.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/721/1*Sjn_GH_j_qP8K7lbZFgCJw.png" /></figure><h3><strong>Components of a message queue:</strong></h3><p>The essential components of message queues are producers, consumers, and a queue. Producers generate the messages, consumers process those, and the queue acts as a buffer to store the messages. Producers and consumers work independently, and we can host them on different servers. The queue connects them both. Applications can have multiple producers and consumers.</p><p>In more advanced message queue systems, a message broker may be present. The message broker acts as an intermediary between producers and consumers, facilitating the routing and delivery of messages. It can provide additional features like message filtering, transformation, and protocol translation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/661/0*FzX7W3fWiWFTbfIl" /><figcaption>Components of a message queue</figcaption></figure><h3><strong>Routing Methods:</strong></h3><p>Different routing methods are available based on the type of broker you use. Some of the most common ones are listed below.</p><h4><strong>Direct Worker Queue Method:</strong></h4><p>In this method, multiple consumers cannot process an identical message. We can have one or more consumers competing for a single message. This method is well-suited for distributing time-consuming tasks across multiple worker machines. For instance, we can use this method to process videos or resize images.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/661/0*EbVoP2TxgdGoC-0b" /><figcaption>Direct Worker Queue Method</figcaption></figure><h4><strong>PubSub Method:</strong></h4><p>In this method, multiple consumers can process an identical message. Consumers subscribe to topics, while producers publish messages about a topic. Consumers receive the subscribed topic messages.</p><p>For example, the E-commerce application can publish a message to a topic each time a purchase is confirmed. You can have a consumer that notifies shipping providers and a different consumer that processes another action.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/761/0*aShL5w--h5yKN2_q" /><figcaption>PubSub Method</figcaption></figure><h4><strong>Custom Routing Rule:</strong></h4><p>Some message brokers may also support different forms of custom routing, where a consumer can decide on messages to consume.</p><p>For example, RabbitMQ can use the concept of binding to create flexible routing rules.Logging and Alerting are good examples of custom routing rules based on pattern matching.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/761/0*oHUUT_av2suT4j8Q" /><figcaption>Custom Routing Rule</figcaption></figure><h3><strong>Benefits</strong>:</h3><h4><strong>Scalability:</strong></h4><p>We can have multiple producers and consumers in our message queues. We can increase our overall throughput by adding more consumers. Also, due to the decoupling of producers and consumers, we do not need to make publisher configuration changes if we add more consumers. Producers do not need to know how many consumers are there or where we host them.</p><h4><strong>Evening out traffic peaks:</strong></h4><p>When a system experiences a sudden spike in traffic or incoming requests, the message queue can absorb and buffer the messages. Instead of overwhelming the receiver or the downstream components, the messages are stored in the queue until the system can process them at a controlled rate. This buffering capability allows the system to handle traffic spikes without compromising performance or causing service disruptions.</p><h4><strong>Isolating Failures and Self-Healing:</strong></h4><p>The separation of producers and consumers helps in isolating failures.</p><p>Components’ failure on either side does not affect the other. On the server failure, you add the replacement, and the system automatically catches up with the queues and the draining messages over time. Instead of breaking the entire application whenever a backend server goes offline, all we experience is reduced throughput.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/699/0*hxwZLxrTD52RJaUX" /><figcaption>Decoupling of Producers and Consumers</figcaption></figure><h3><strong>Message Queue Related Challenges:</strong></h3><h4><strong>No message Ordering:</strong></h4><p>The order of how messages arrive and where they get consumed can result in inconsistent and unknown behaviors. If the order of operations of messages does matter in achieving the desired result, then we seriously need to consider the order of tasks. We can ensure single consumer consumes related messages in sequence. We can use features like FIFO queues, partial message ordering, or enforcing constraints on our application logic to mitigate the ordering issue.</p><h4><strong>Message Requeuing:</strong></h4><p>In failure scenarios, messages can get re-queued. Dealing with this problem can be easy or difficult, depending on the application’s needs.</p><p>One of the strategies worth considering is to employ at least one delivery instead of exactly one delivery. By allowing messages to be delivered to your consumers more than once, you make your system more robust and reduce constraints put on the queue itself and its workers. But one must ensure that the consumption of the same message twice does not affect the requirement of a system.</p><h4><strong>Race Conditions:</strong></h4><p>One of the significant challenges related to asynchronous processing is guaranteeing the order of related operations. The results of different messages can arrive at separate times. The order of messages results depends on various factors and is prone to race conditions.</p><h3><strong>Message Queue Related Anti Patterns:</strong></h3><h4><strong>Treating the Message Queue as a TCP socket:</strong></h4><p>Some brokers allow you to create a return channel, a path to send messages back to the producer. If you use this a lot, you might end up with an application that is more asynchronous than synchronous.</p><p>Ideally, messages need to be fire and forget requests. Opening a response channel and waiting for the response of messages makes messaging components more coupled. When building a scalable system, avoid using return channels.</p><h4><strong>Treating the Message Queue as a Database:</strong></h4><p>You should not allow random access to elements of the queue. Also, not allow deleting or updating them. It may prevent you from scaling out and migrating to a different message broker.</p><h4><strong>Coupling Message Producers with Consumers:</strong></h4><p>It is a good practice to avoid introducing explicit dependency between producers and consumers. It is better to think message body as a contract. Messages should not have logic or some code within that introduces some dependencies.</p><h4><strong>Lack of Poison Message Handling:</strong></h4><p>Consumers might not process all the messages. The application needs to have a way to handle these failed cases. We can deal with poison messages differently depending on the message broker. Dead letter queues and automatic removal of messages after some retries are some of the options.</p><h3>Choosing the Perfect Message Queue:</h3><p>The message queue landscape offers a variety of options that are widely used in the industry. Here are some popular message queue systems and technologies along with considerations to help you choose the right one for your needs:</p><h4>RabbitMQ:</h4><ul><li>RabbitMQ is a mature and feature-rich open-source message broker.</li><li>It supports multiple messaging patterns, including point-to-point, publish-subscribe, request-reply, and more.</li><li>RabbitMQ is known for its flexibility, robustness, and wide language support.</li><li>Consider using RabbitMQ if you require a reliable and highly customizable message broker with broad community support.</li></ul><h4>Apache Kafka:</h4><ul><li>Kafka is a distributed streaming platform that can function as a message queue.</li><li>It is designed for high-throughput, fault-tolerant, and real-time streaming applications.</li><li>Kafka provides strong durability and fault-tolerance guarantees, making it suitable for use cases that require processing large volumes of data in real-time.</li><li>Consider Kafka if you need a scalable and fault-tolerant messaging system for handling high volumes of data streams and building real-time data pipelines.</li></ul><h4>ActiveMQ:</h4><ul><li>ActiveMQ is an open-source, Java-based message broker that supports various messaging protocols.</li><li>It provides features such as message persistence, message acknowledgment, and message filtering.</li><li>ActiveMQ is suitable for integrating different systems and applications using different messaging patterns.</li><li>Consider ActiveMQ if you prefer a lightweight, Java-based message broker with support for multiple messaging protocols.</li></ul><h4>Amazon Simple Queue Service (SQS):</h4><ul><li>SQS is a fully managed message queue service provided by Amazon Web Services (AWS).</li><li>It offers reliable and scalable message queuing with features such as automatic scaling, message retention, and dead-letter queues.</li><li>SQS is a good choice if you are already using AWS services and require a fully managed solution without the need for self-hosting or maintenance.</li></ul><h4>Google Cloud Pub/Sub:</h4><ul><li>Pub/Sub is a managed messaging service provided by Google Cloud Platform (GCP).</li><li>It offers reliable and scalable messaging with features like push and pull delivery, topic-based publish-subscribe model, and event-driven architecture.</li><li>Pub/Sub is suitable for building real-time analytics, event-driven systems, and streaming pipelines on GCP.</li></ul><p>When choosing the right message queue, consider the following factors:</p><ul><li><strong>Scalability</strong>: Determine if the message queue can handle the expected message throughput and scale horizontally as your application grows.</li><li><strong>Durability and Reliability:</strong> Consider the guarantees provided by the message queue in terms of message persistence, delivery guarantees, and fault tolerance.</li><li><strong>Messaging Patterns: </strong>Assess if the message queue supports the messaging patterns required by your application, such as point-to-point, publish-subscribe, request-reply, etc.</li><li><strong>Integration and Language Support:</strong> Check if the message queue has libraries and SDKs available for the programming languages and frameworks you are using.</li><li><strong>Operational Overhead:</strong> Evaluate the ease of deployment, configuration, monitoring, and maintenance of the message queue.</li><li><strong>Community and Support:</strong> Consider the size and activity of the community around the message queue, as well as the availability of documentation, tutorials, and support channels.</li></ul><h3>Final Thoughts:</h3><p>Message queues have emerged as a crucial solution for enabling efficient and seamless communication in modern application architectures. They address the challenges of distributing tasks, coordinating between components, and ensuring reliable messaging in distributed systems.</p><p>By leveraging message queues, developers can achieve scalability by adding multiple producers and consumers. The decoupling of producers and consumers allows for independent scaling without the need for configuration changes. Additionally, message queues help in handling traffic spikes by buffering messages and controlling the rate of processing, preventing overload and service disruptions.</p><p>One of the significant advantages of message queues is their ability to isolate failures. With producers and consumers working independently, failures on one side do not impact the other. The system can recover by replacing failed components and catching up with the messages in the queue, resulting in reduced downtime and improved reliability.</p><p>However, message queues also come with their own set of challenges. Ensuring message ordering, dealing with message re-queueing, and managing race conditions require careful consideration and appropriate mitigation strategies. Developers must design their systems to handle these challenges effectively and maintain the desired order and consistency.</p><p>Furthermore, it’s important to avoid anti-patterns such as treating the message queue as a TCP socket or a database, and coupling message producers with consumers. By adhering to best practices and considering message queue design principles, developers can ensure loose coupling, scalability, and maintainable messaging systems.</p><p>In conclusion, message queues offer a powerful mechanism for enabling asynchronous processing, seamless communication, and efficient coordination within distributed systems. By understanding their benefits, challenges, and best practices, developers can leverage message queues effectively to build scalable, reliable, and responsive applications in today’s dynamic software landscape.</p><p>If you liked this article don’t hesitate to applaud it! Thank you.</p><h3>Sources:</h3><ul><li>Artur Ejsmont. Web Scalability for Startup Engineers McGraw Hill LLC, 2015:New York.</li><li><a href="http://draw.io/">Draw.io</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=372ebcc9865b" width="1" height="1" alt=""><hr><p><a href="https://medium.com/red-buffer/demystifying-message-queues-seamless-communication-for-asynchronous-processing-372ebcc9865b">Demystifying Message Queues: Seamless Communication for Asynchronous Processing</a> was originally published in <a href="https://medium.com/red-buffer">Red Buffer</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>