<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by siddhesh jog on Medium]]></title>
        <description><![CDATA[Stories by siddhesh jog on Medium]]></description>
        <link>https://medium.com/@siddhesh.jog?source=rss-95a776206365------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*JZBFDxkSnQ2z9nZ9bZ8PoA.png</url>
            <title>Stories by siddhesh jog on Medium</title>
            <link>https://medium.com/@siddhesh.jog?source=rss-95a776206365------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 11:46:08 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@siddhesh.jog/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[The Interstellar Blunder and the Larger question of Trust & Sacrifice in modern day life]]></title>
            <link>https://medium.com/age-of-awareness/the-interstellar-blunder-and-the-larger-question-of-trust-in-modern-day-life-1f298164648e?source=rss-95a776206365------2</link>
            <guid isPermaLink="false">https://medium.com/p/1f298164648e</guid>
            <category><![CDATA[philosophy]]></category>
            <category><![CDATA[trust]]></category>
            <category><![CDATA[sacrifice]]></category>
            <category><![CDATA[character]]></category>
            <category><![CDATA[ethics]]></category>
            <dc:creator><![CDATA[siddhesh jog]]></dc:creator>
            <pubDate>Sat, 15 Nov 2025 19:12:15 GMT</pubDate>
            <atom:updated>2025-11-19T08:34:19.284Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dLD_zkehWPXP4uJl2Wxbsg.png" /><figcaption>The Interstellar Blunder and the Larger question of Trust in modern day life</figcaption></figure><p><strong>Warning: This post tears apart the Interstellar ending. Major spoilers ahead!</strong></p><p>Now that I’ve pulled you in with a click bait, let me start by saying that I love the movie Interstellar. Well, I love most things about it. Some of it is sequences like the Docking sequence will live long in my memory.</p><blockquote>CASE: “Cooper it’s not possible!” COOPER: “No it’s necessary!”</blockquote><p>Contrary to many opinions I do actually like the “Dr Mann being a coward and evil!!” subplot. While I don’t love how it was telegraphed early in the film (“Nature is not evil, whatever evil exists we take with ourselves”), that hardly qualifies as a blunder.</p><h4>The Blunder Defined: When Perfect Proof Ruins the Story</h4><p>The blunder is how Interstellar ends. Lets recap the ending a bit.</p><p>Cooper decides to enter the black hole for two reasons. To help Dr Brandt escape the Black hole’s gravity by shedding weight. And also to give his daughter and hundreds of thousands of people back home a chance to survive by perhaps getting information about the singularity from the blackhole.</p><p>As he dives inside the blackhole he is rescued by the “beings” (who are essentially future humans) and put inside a tesseract, which is essentially Murph’s bedroom for all time that it exists. This is because while the future beings are able to traverse time, they are not bounded by the same mechanisms that we are and hence unable to communicate with Murph.</p><p>Only Cooper, with his pure fatherly love for his daughter can find a way to communicate with her. He does find a way. He sends her the required information through Morse code encoded in the minute hand of the watch that he gifted her when he left earth. (Side note: I love how Morse was inserted into our minds earlier in the film so we are already expecting Cooper to use Morse by this point).</p><p>Then the tesseract closes. And then (no no no no no, please no ) Cooper is transported back through the worm hole near Saturn. He is intercepted by humans and flown to Cooper station. There he meets his daughter to confirm her “belief” that it was him (<strong>dad!!)</strong> that helped her all along. Only to then go back out of Cooper station to find “Dr Brandt”.</p><p>This forced happy ending. This is the blunder.</p><h4>The Blighted Victory: The Cost of Confirmation on Character and Sacrifice</h4><p>Think about it, the secret of the singularity that allowed Murph to harness the power of gravity was sent to her in Morse code. What was the starting point of that message? How did Murph even figure out where the sequence starts? Murph obviously had to work very hard to figure out the message. Maybe weeks even months. Then she would have had to harness that information for figuring out how to lift such large “stations” off the surface of the earth. Her effort here is nothing short of heroic. The final meeting reframes the climax as Cooper’s successful delivery rather than Murph’s successful reception and application. By confirming everything, the film shifts the emotional weight from Murph’s work to Cooper’s paternal fulfilment.</p><p>Then there is the even more important aspect of trust and belief in other people. As Murph figured out the secret to get people off the planet in large numbers, there would have been days when she would have been plagued with doubt. Where she would have questioned everything:</p><blockquote>“Did dad really come through? Was it a coincidence? Did he leave us to die?”</blockquote><p>Days when her belief that it was her dad who gave her the answers, would have been tested. Days of intense tests of character where she pulled through. By having Cooper and Murph meet the flim breaks that belief and establishes empirical evidence where none is needed. The beauty of human relations truly shines through when the trust between people is implicit, it takes character and a lot of hard work to establish such trust. The doubt about whether her dad truly did come through for her, was the fire that purified Murph’s endevours. By closing the loop the film chooses the safe, contractual relationship model over the messy, challenging, and ultimately more profoundly human model of implicit trust.</p><p>The blunder doesn’t just cheapen Murph’s success; it compromises the integrity of <strong>Cooper’s sacrifice</strong>. All through the film, Cooper is seen making every decision to secure his children’s future, even if it means being misunderstood by the very children he is trying to save. He is making a permanent, ultimate choice — accepting death and separation. This definitive action, undertaken <em>without</em> the expectation of reunion or reward, represents the purest form of paternal love and sacrifice. By resurrecting him and granting him the sentimental rewards of meeting Murph and leaving immediately to find Dr. Brandt, the film <strong>undercuts the nobility of his original choice</strong>.</p><p>This cinamatic betrayal of Cooper’s sacrifice and Murph’s faith is painful because it mirrors what we witness every day in the real world: the relentless push to replace implicit character based trust, with explicit documented contracts. This erosion of trust manifests everywhere, from our workplaces to our most intimate relationships, transforming emotional bonds into mere transactions.</p><h4>The Corporate Parallel: When Confirmation Steals the Credit</h4><p>The corporate world is replete with such examples. An employee works super hard for a year. Their work is appreciated by all in the team, and everyone acknowledges that without their contributions, things may have gone south. Yet, when the employee is finally rewarded after appraisal with a raise or a promotion, the manager lets slip:</p><blockquote>“Do you know how hard I had to work for this? They were not ready to give you this promotion. I worked day in and day out to ensure you got it.”</blockquote><p>This unnecessary “closing of the loop,” while it may indeed be true, serves only to diminish the effort of the employee in their own mind. However hard I work, whatever I do, it was really the manager who fought through for my promotion. Crucially, this over-explanation also reveals a lack of implicit trust from the manager — a failure to believe the employee understands, without being told, that their manager has their back. The manager feels compelled to provide <em>empirical proof</em> of their support, thereby turning the promotion from a reward for the employee’s work into a receipt for the manager’s effort.</p><h4>The Age of the Receipt: From Implicit Marriage to Explicit Agreement</h4><p>The erosion of trust is even more visible in personal life. We are living in The Age of the Receipt, where the messy, hard-won virtue of trust is being systematically undermined. Consider the shift from a traditional marriage, built on the delicate, implicit promises of shared future and mutual faith, to the legal finality of a prenuptial agreement. While legally sound, the <em>spirit</em> of the prenup is one of protection against betrayal, an insurance policy against character failure. It is a document that replaces the belief in the other person with a binding contract. This mindset insists on quantifying risk and demanding guarantees, but in doing so, it strips the relationship of its most profound quality: the courage required to be vulnerable without a safety net. Just as Cooper’s arrival cheapened Murph’s faith, our modern obsession with security cheapens our relationships by demanding proof where the greatest value lies in the unconfirmed, yet freely given, belief.</p><h4>The True Altruist: When Acknowledgment Undermines the Gift</h4><p>The cheapening of Cooper’s sacrifice finds a disturbing parallel in the modern trend of <strong>performative altruism and virtue signalling</strong>. Consider the parent who meticulously documents their efforts to raise “strong, independent girls” or the individual who posts long, philosophical treatises about charitable giving, all under the guise of sharing knowledge or inspiring others. <strong>The act itself — the quiet, difficult work of effective parenting, or the selfless act of charity — is profound.</strong> But when the camera is flipped, and the effort is framed not as a private duty but as a <strong>public message</strong>, the primary goal shifts from <strong>doing good</strong> to <strong>being <em>seen</em> doing good</strong>. Just as Cooper’s final moments were pulled from the sublime privacy of sacrifice and placed onto the public stage of the Cooper Station for a sentimental reward, the purity of the real-world act is reduced. The reward becomes the <strong>social affirmation and validation</strong> gained from the post, effectively undermining the noble, unacknowledged burden that was the original gift.</p><p>Ultimately, <em>The Interstellar Blunder</em> is not just a problem with a movie, but a reflection of a deep shift in societal values. While we might see the film’s ending as simply satisfying a plot requirement, we forget that the foundations of civilisation — from pioneering journeys to the foundational bonds of community — were built on implicit trust. This virtue is the currency of high character. In past eras, faith and unspoken contracts were the pillars of human interaction. By insisting on closing the loop and giving Murph empirical proof, the film validates our modern slide toward the easy answer: the receipt instead of the risk. But the risk — the fire of doubt that refined Murph’s efforts — is precisely what makes relationships, and achievements, profoundly meaningful.</p><p>I suppose in the larger scheme of things all we can do is reflect on: <strong>What valuable human character trait is lost when we prioritise the certainty of explicit contract over the strength required for unconfirmed, yet freely given, trust?</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1f298164648e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/age-of-awareness/the-interstellar-blunder-and-the-larger-question-of-trust-in-modern-day-life-1f298164648e">The Interstellar Blunder and the Larger question of Trust &amp; Sacrifice in modern day life</a> was originally published in <a href="https://medium.com/age-of-awareness">Age of Awareness</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AWS Layer 4 Network & Connectivity Primer]]></title>
            <link>https://medium.com/@siddhesh.jog/aws-layer-4-network-connectivity-primer-c862e7cc8b67?source=rss-95a776206365------2</link>
            <guid isPermaLink="false">https://medium.com/p/c862e7cc8b67</guid>
            <category><![CDATA[transit-gateway]]></category>
            <category><![CDATA[vpc]]></category>
            <category><![CDATA[shared-vpc]]></category>
            <category><![CDATA[multi-region]]></category>
            <category><![CDATA[network]]></category>
            <dc:creator><![CDATA[siddhesh jog]]></dc:creator>
            <pubDate>Tue, 15 Feb 2022 04:47:41 GMT</pubDate>
            <atom:updated>2022-02-15T04:47:41.773Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tYDx8JwCgyumHiYNXmFl8A.png" /></figure><p>Whether you are a developer learning the ropes, or a startup finding its feet, or a cost and time conscious SMB or an enterprise with a <a href="https://www.morningstar.com.au/Video/the-morningstar-economic-moat-rating/2593">wide moat</a>, hosting workloads in AWS requires setting up network and connectivity components.</p><p>And this constitutes (strictly from a Layer 4 perspective):</p><ul><li>Setting up a private network in the form of one or many VPCs.</li><li>Facilitating connectivity between workloads within the network.</li><li>Extending connectivity from cloud to on-premises.</li><li>Allowing remote access to cloud workloads.</li></ul><p>There is also the matter of <em>connectivity to AWS services</em> as well as <em>Layer 7 Connectivity</em> but that demands a post on its own.</p><p>In this post we will look at the aforementioned layer 4 connectivity tasks that help us setup a private network and the bare bones connectivity on which everything else relies. All diagrams are drawn without explicitly display availability zones; so please do assume that networks are spread across all availability zones in a region.</p><p>Where do you start though? Well, you start with the basics and then progressively add improvements to your design.</p><h3>Cloud Network</h3><h4>Start Simple</h4><p>At its simplest an account is the smallest unit of organisation in AWS. Defined as “a container for your AWS resources”; a single account allows you to get started with your AWS setup.</p><p>In its simplest form a network constitutes a single VPC with multiple subnets to host all of your public and private workloads.</p><p>The internet gateway provides two way connectivity to the internet, the NAT gateway provides egress only internet access for Private workloads.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/643/1*caztsi_8S4_TVYDhSjOAMA.jpeg" /></figure><pre><strong>Pros</strong></pre><pre><strong>√</strong> A setup that is as simple as it gets.</pre><pre><strong>Cons</strong></pre><pre><strong>x</strong> All workloads in the same VPC creates a huge blast radius. A compromised VPC will mean all workloads may be vulnerable.</pre><pre><strong>x </strong>Fine grained Role based access control is required which can be a maintenance challenge.</pre><pre><strong>x </strong>You may hit the account quota limits.</pre><h4>Contain Blast radius: Segregate environments</h4><p>The first improvement you could make is to have separate VPCs for separate environments/business units. The VPCs are still hosted in the same account disconnected from one another.</p><p>Each VPC has its own internet gateway and NAT Gateway for connectivity.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/891/1*tHSRPvwnDhkzGTqR9wll3g.jpeg" /></figure><pre><strong>Pros</strong></pre><pre><strong>√</strong> Simple setup.</pre><pre><strong>√ </strong>Containment against compromised network.</pre><pre><strong>Cons</strong></pre><pre><strong>x</strong> Disconnected workloads across teams.</pre><pre><strong>x </strong>Fine grained Role based access control is required which can be a maintenance challenge.</pre><pre><strong>x </strong>You may hit the account quota limits.</pre><h4>Inter VPC routing: Add Connectivity</h4><p>With separate VPCs for hosting and managing workloads belonging to different teams/environments in the organisation there will be a need to connect these workloads.</p><p>In its easiest form this can be achieved by VPC peering. Since peering is not transitive the VPCs will continue to have their own internet and NAT gateways.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/891/1*e64CFwqxkMgF3d9XW5qiNw.jpeg" /></figure><pre><strong>Pros</strong></pre><pre><strong>√</strong> Simple setup.</pre><pre><strong>√ </strong>Containment against compromised network.</pre><pre><strong>√ </strong>Connectivity between cross team/department workloads.</pre><pre><strong>Cons</strong></pre><pre><strong>x</strong> VPC peering is neither scalable nor transitive.</pre><pre><strong>x </strong>Fine grained Role based access control is required which can be a maintenance challenge.</pre><pre><strong>x </strong>You may hit the account quota limits.</pre><h4>Organise : Deploy to multiple accounts</h4><p>Even if you are only just learning AWS, a more palatable approach is to use a multi account setup.</p><p>You may create one account per environment like: dev, test, integration, sandbox, pre-prod, prod. As well as account for individual teams within the organisation: payments, digital experience etc.</p><p>You can then host multiple VPCs, 1 each in these accounts to host your workloads.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/891/1*GDVRgdziVKZksfB9uF0NSQ.jpeg" /></figure><pre><strong>Pros</strong></pre><pre><strong>√</strong> Excellent containment against blast radius with multi account deployment.</pre><pre><strong>√ </strong>Containment against compromised network.</pre><pre><strong>√ </strong>Comparatively coarse grained RBAC.</pre><pre><strong>√ </strong>Account quota limit not a problem in most scenarios.</pre><pre><strong>Cons</strong></pre><pre><strong>x</strong> VPC peering is neither scalable nor transitive.</pre><pre><strong>x </strong>No reuse of ingress &amp; egress points, difficult to apply consistent security policy.</pre><pre><strong>x </strong>Complex setup.</pre><h4>Scalable connectivity: Multi VPCs with Transit Gateway</h4><p>VPC peering does not scale and can be difficult to maintain. For every n VPCs we require n * (n-1)/2 connections. For 10 VPCs thats a whopping 45 connections.</p><p>The multi account multi VPC, setup can be further improved by replacing the VPC peering with transit gateway.</p><p>Not only does this simplify connectivity with the hub and spoke model (<em>the hub being the transit gateway</em>), it also lends itself nicely to reducing duplication since the transitive nature of connectivity allows creation of single ingress/egress VPC.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xzS5iR4BXLcbMOv5RPSUwA.jpeg" /></figure><pre><strong>Pros</strong></pre><pre><strong>√</strong> Excellent containment against blast radius with multi account deployment.</pre><pre><strong>√ </strong>Scalable connectivity solution to accomodate large number of VPCs.</pre><pre><strong>√ </strong>Comparatively coarse grained RBAC.</pre><pre><strong>√ </strong>Account quota limit not a problem in most scenarios.</pre><pre><strong>√ </strong>Reuse of ingress and egress allows for application of security policy in a consistent manner.</pre><pre><strong>Cons</strong></pre><pre><strong>x</strong> N/w component hosted in workloads accounts, making RBAC complex.</pre><pre><strong>x </strong>Complex setup.</pre><h4>Separation of Concerns: Separately hosted TGW</h4><p>As the number of workloads grow with more environments (Dev, Staging, Pre-prod, Prod) or more teams (analytics, digital, integrations) or business units (consumer, retail etc), so does the need to provide robust and standardised network and connectivity services across the organisation.</p><p>You might have a dedicated team that is responsible only for managing network and connectivity.</p><p>The logical next step then is to separate out the connectivity components into their own account.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oEEZMH0G79OFHZO6otp69g.jpeg" /></figure><pre><strong>Pros</strong></pre><pre><strong>√</strong> Excellent containment against blast radius with multi account deployment.</pre><pre><strong>√ </strong>Scalable connectivity solution to accomodate large number of VPCs.</pre><pre><strong>√ </strong>Comparatively coarse grained RBAC.</pre><pre><strong>√ </strong>Account quota limit not a problem in most scenarios.</pre><pre><strong>√ </strong>Reuse of ingress and egress allows for application of security policy in a consistent manner.</pre><pre><strong>√ </strong>Separation of concerns. N/w components hosted independently.</pre><pre><strong>Cons</strong></pre><pre><strong>x </strong>Very Complex setup. Requires deployment of multiple AWS solutions (STNO, Control Tower Customisation).</pre><h4>Flip it on Its head: The shared VPC</h4><p>With all that work we’ve finally got a network setup that allows for multiple environments and workloads.</p><p>We can of course flip this totally on its head by using the AWS Shared VPC. Made available in 2019, this feature allows sharing of VPC subnets using the Resource Access Manager. This allows creating a single VPC either for your entire organisation; or you may create one per network domain for enterprises.</p><p>What I truly love about the shared VPC is that it objectively demonstrates the “listening to the customer’s needs” part that AWS always talk about. So kudos to them on this.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nzlTn8UOXG152CKmnWx0Rg.jpeg" /></figure><pre><strong>Pros</strong></pre><pre><strong>√</strong> Containment against blast radius if individual subnets are provided to customers.</pre><pre><strong>√ </strong>Out of the box cross team/environment connectivity.</pre><pre><strong>√ </strong>Coarse grained RBAC with account separation.</pre><pre><strong>√ </strong>Account quota limit not a problem in most scenarios.</pre><pre><strong>√ </strong>Reuse of ingress and egress </pre><pre><strong>√ </strong>Separation of concerns. N/w components hosted independently.</pre><pre><strong>√ </strong>Easy Setup.</pre><pre><strong>Cons</strong></pre><pre><strong>x </strong>Subnet isolation if required, need usage of NACLs and creates a complex setup.</pre><pre><strong>x </strong>All workloads hosted in one VPC, without subnets can create room of attacks that starve the VPC for IPs.</pre><h3>Go Global</h3><h4>Transit Gateway peering</h4><p>Globe trotting enterprises will have workloads in different regions across the world. AWS introduced inter region peering in 2019 to allow connecting two Transit gateways across regions. Even if you used the shared VPC model you would need Transit gateways to setup in each region to connect shared VPCs across regions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2PGbpBmEG8Qzm3_zblLdug.jpeg" /></figure><pre><strong>Pros</strong></pre><pre><strong>√</strong> Global connectivity.</pre><pre><strong>Cons</strong></pre><pre><strong>x </strong>Complex setup with multiple ingress-egress.</pre><h3>On Premises &amp; Remote</h3><h4>Orchestration Only: Setup Site to Site VPN</h4><p>Thus far we’ve been able to establish a robust multi account “private” cloud network.</p><p>With on-premises networks there is a need to establish a private connection with your cloud environment.</p><p>In case you only have a private network used by employees to orchestrate the cloud or have <em>very few</em> on-premises workloads a VPN connection is the way to go.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RktQ0z8VXut_NaCcOTCwYA.jpeg" /></figure><pre><strong>Pros</strong></pre><pre><strong>√</strong> Seamless connectivity with on-premises/office network.</pre><pre><strong>Cons</strong></pre><pre><strong>x </strong>Connection does not scale well for large workloads.</pre><h4>On Premises Workloads: Use Direct Connect</h4><p>For enterprises; bulk of their workloads are on premises data centres. VPN connectivity will not scale well for connecting workloads in the cloud to those in the on premises data centre.</p><p>Therefore, in order to create seamless connectivity between cloud and on premises workloads you <em>MUST</em> setup a direct connect connection.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vmN4-5ASSSf5qQ8l7wVWFw.jpeg" /></figure><pre><strong>Pros</strong></pre><pre><strong>√</strong> Seamless connectivity with on-premises/office workloads and scales well for 100s of workloads.</pre><pre><strong>Cons</strong></pre><pre><strong>x </strong>Involves significant hardware setup and costs. Suitable only for enterprises.</pre><h4>Connect from Home: Client VPN &amp; VPN Client</h4><p>Almost everyone is working from home and doing a fair job of it. You may have an existing solution to connect to your on premises network which coupled with Site to Site VPN can provide access to your workloads in AWS.</p><p>If not, you may use the Client VPN setup to allow access to your VPCs and workloads from employee machines.</p><p>AWS Client VPN allows you to setup such a connection, alternatively you may choose to you your own VPC on an EC2 instance.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FAk02fTwIG-teu7pxWEa0Q.jpeg" /></figure><h3>Other things to consider</h3><p>The sections above have covered only setup and connectivity. But you are far from done.</p><p>You then have to secure your workloads, secure your ingress and egress routes, setup IP address management to ensure that N/W CIDR allocation is done appropriately, add monitoring capability and much more.</p><p>AWS has dedicated services and features for the same:</p><p>For network &amp; workloads security you have the NACLs, Security groups.</p><p>For securing Ingress and Egress you have AWS Shield Advanced and Network Firewall.</p><p>For IPAM there is the recently released VPC IPAM feature.</p><p>For monitoring you may switch on VPC flow logs and monitor VPC, transit gateway and direct connect events and metrics in CloudWatch.</p><p>There is of course the Layer 7 DNS setup and connectivity for individual teams and their workloads which deserves a full article in its own right. But that is covered by Route 53 Hosted zones and resolvers.</p><p>Finally there is Layer 7 security which is addressed by AWS WAF among other tools.</p><h3>How do you choose</h3><p>This is not a straight forward decision. You may be a small business today but on your way to becoming a future enterprise, you may be large business with a small technology presence (<em>although this is increasingly not the case</em>).</p><p>At the very least irrespective of size it is a good idea to have multiple accounts and thus a network spread across multiple accounts.</p><p>From there on it comes down to a set of decisions based on specific circumstances around:</p><ul><li>number of teams</li><li>number of environments</li><li>tolerance for blast radius</li><li>ease of setup and operations</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6J_qavb5abZ4h2oet0oICA.jpeg" /></figure><p>I’ve tried to capture here some key decisions above but recommend that each team ask its own set of questions to come up with their ideal network setup. But this my know means is an exhaustive list of questions to ask; so tailor according to your needs as AWS has almost all based covered.</p><p><em>Thank you and Happy networking!</em></p><h3>References</h3><ul><li><a href="https://docs.aws.amazon.com/vpn/latest/s2svpn/your-cgw.html">Your customer gateway device</a></li><li><a href="https://docs.aws.amazon.com/vpn/latest/s2svpn/how_it_works.html">How AWS Site-to-Site VPN works</a></li><li><a href="https://aws.amazon.com/blogs/networking-and-content-delivery/building-an-egress-vpc-with-aws-transit-gateway-and-the-aws-cdk/">Building an egress VPC with AWS Transit Gateway and the AWS CDK | Amazon Web Services</a></li><li><a href="https://aws.amazon.com/blogs/networking-and-content-delivery/creating-a-single-internet-exit-point-from-multiple-vpcs-using-aws-transit-gateway/">Creating a single internet exit point from multiple VPCs Using AWS Transit Gateway | Amazon Web Services</a></li><li><a href="https://aws.amazon.com/about-aws/whats-new/2021/12/amazon-virtual-private-cloud-vpc-announces-ip-address-manager-ipam/">Amazon Virtual Private Cloud (VPC) announces IP Address Manager (IPAM) to help simplify IP address management on AWS</a></li><li><a href="https://docs.aws.amazon.com/vpc/latest/userguide/vpc-sharing.html">Share your VPC subnets with other accounts</a></li><li><a href="https://docs.aws.amazon.com/vpc/latest/tgw/tgw-getting-started.html">Tutorials: Get started with AWS Transit Gateway</a></li><li><a href="https://aws.amazon.com/blogs/networking-and-content-delivery/building-a-global-network-using-aws-transit-gateway-inter-region-peering/">Building a global network using AWS Transit Gateway Inter-Region peering | Amazon Web Services</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c862e7cc8b67" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Litmus Testing your micro services]]></title>
            <link>https://medium.com/@siddhesh.jog/litmus-testing-your-micro-services-4009002a9018?source=rss-95a776206365------2</link>
            <guid isPermaLink="false">https://medium.com/p/4009002a9018</guid>
            <category><![CDATA[testability]]></category>
            <category><![CDATA[ease-of-deployment]]></category>
            <category><![CDATA[microservices]]></category>
            <category><![CDATA[rest]]></category>
            <category><![CDATA[reliability]]></category>
            <dc:creator><![CDATA[siddhesh jog]]></dc:creator>
            <pubDate>Mon, 17 Jan 2022 23:19:35 GMT</pubDate>
            <atom:updated>2022-01-17T23:19:35.426Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qfH9-nialx_1kXjPGilXvw.jpeg" /><figcaption>Is it a well defined micro service?</figcaption></figure><h3>What is a micro service?</h3><p>Sam Newman, in his excellent book: <a href="https://samnewman.io/books/building_microservices/"><em>Building Micro services</em></a>, defines micro services as small autonomous services that work together. Micro services take the single responsibility principal to the service level. By definition, on account of being autonomous, micro services are deployed on separate machines (<em>although what constitutes a separate machine is a whole another discussion</em>). Thus inter service communication is via network calls.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*c6krB2LfYTpV-reEN6ADvw.png" /></figure><blockquote>While this setup does bring with it complexities; the explicit “machine” boundary for enforcing autonomy <strong>enables the possibility</strong> of a whole host of benefits.</blockquote><h3>Benefits of adopting Micro services architecture</h3><p>Firstly; micro services are not for everyone. We will need an entire article dedicated to why that is so.</p><p>But for organisations that <strong><em>can adopt</em> </strong>micro services; there is a possibility of realising a whole host of benefits such as:</p><h4>Improved Resilience</h4><p>Well defined micro services do <em>one thing and one thing only. </em>each service acts on a defined set of inputs and produces a defined set of outputs. This can allow us to build systems that can isolate problems within a service with the rest of the services carrying on as usual.</p><h4>Easier scaling</h4><p>With the platform split into smaller components; these components can be differentially scaled. This can also enable cost optimisation.</p><h4>Faster ideation to implementation</h4><p>On account of clear service demarkation changes can be implemented faster Turn around time on the <em>implement-inspect-adapt </em>feedback loop can be greatly reduced, which has a direct impact on business outcomes.</p><h4>Freedom of Technological Choice</h4><p>With the platform split into smaller components; these components can be built using different technologies suitable for the specific task.</p><p>Notice though that I’ve use phrases such as “possibility of”, “can be” when discussing the benefits. That is because while micro services can provide these benefits; the benefits themselves are not intrinsic qualities of micro services.</p><p>More often than not incorrect implementation strategies can derail the whole platform. There is a plethora of articles going around announcing why micro services should not be used. While not all of them are incorrect or unfounded, quite a few of the issues that teams encounter are due to incorrect implementation.</p><blockquote>Benefits are only realised if the micro services are well defined.</blockquote><h3>Litmus testing for a “Well defined Micro Service”</h3><p>First instinct when designing and building micro services is to stand up some APIs by roughly decomposing the platform domain. Having defined some services it is still quite difficult to understand what is a well designed service.</p><p>We can come up with a list of “must haves” for what constitutes a “well defined micro service”.</p><blockquote>However, I find it useful to think of a well defined micro service in terms of the problems it MUST solve.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*58X4Qiq--saMA_syj_-qZw.png" /><figcaption>You may enter the “well designed micro service” realm by solving for these key problems</figcaption></figure><p>For your service to be “well defined” it must solve for the problems depicted above. These problems thus serve as somewhat of a litmus test.</p><h4>Are you able to deploy your service without affecting another team?</h4><p>The test of deployability shines a light on how coupled your services are, be it from a code, test, team or infrastructure perspective. I have deliberately left “independently deployable” out. It does not go far enough.</p><p>If you cannot deploy a service without affecting another or without having to deploy another; your service boundaries are not well defined.</p><p>This can happen if:</p><ul><li>Two services are build within the same component, thus sharing a run time process.</li><li>state of one service is tightly coupled to that of another.</li></ul><p>Independent services must not share a system process.</p><p>For daisy chained or coupled services this can be solved either by ensuring that interface changes are always versioned (in case of REST) or by using a technology such as GraphQL, that allows for a change in the interface (not always, be careful) while remaining backward compatible.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cz1_KoPoQ3IfyC8O8oQBFA.png" /></figure><blockquote>Micro services must be deployable with 0 effect on other teams to allow teams to work independently of one another.</blockquote><h4><strong>Are you able to test your service without needing to work with another?</strong></h4><p>This test reveals how tightly coupled your services are to another from a data exchange stand point.</p><p>One of the key advantages of micro services is to be able to test a set of functionality, represented by the micro service, independently of any other testing. This is critical to easily automate end to end testing for the service and in turn build confidence in releasing code to production continuously.</p><p>You may not be able to test you service independently if:</p><ul><li>A service relies on data outside of its domain boundary for creating /updating its own state.</li><li>A service in a daisy chain relies on functionality outside its boundary directly to produce its own output.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FNZxMNT99gDbTGNNK2Otcw.png" /></figure><p>Independent testability requires the concept of “bounded context” to be understood and applied with rigour.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*syguB3JUhIwWiASunzIghg.png" /></figure><blockquote>A service must be fully responsible for its functionality to ensure that it can be tested independently of another.</blockquote><h4>Are you able to modify your data model without worrying about the impact to another service?</h4><p>This test reveals how segregated on intertwined your data model is and subsequently how easy or difficult it is to modify your model.</p><p>Data separation can be very difficult to maintain in a fast paced environment. Data modelling and normalisation techniques can lead to intertwined data dependencies.</p><p>This can happen if:</p><ul><li>tables from different services are interlinked by means of constraints.</li><li>service reads data from tables belonging to another service.</li></ul><p>Micro services architecture is opinionated about how data pertaining to a service should be stored. It prescribes that each service should maintain its data privately in its own database. The ‘own database’ here is a logical construct which may indeed be a separate database (preferable and possible with services like DynamoDB, MongoDB) or an independent set of tables.</p><p>Intertwined data models can create change and test cross team dependencies which in turn will hamper speed with which new functionality can be implemented does directly affecting a businesses ability to serve its customers.</p><blockquote>A service must own its data model completely.</blockquote><h4>Are you able to trace errors across services?</h4><p>This test reveals if your platform as a whole is observable. How easy or difficult it is to investigate, understand and resolve issues; thus impacting how soon you can address customer grievances.</p><p>In a typical business flows a number of micro services combine to deliver functional capabilities. If an error occurs during one such flow, it can become quite difficult to trace such error across services and relate them to a common request.</p><p>Error analytics and traceability is a must for a well defined micro service architecture. The real world benefit can be a direct impact on perception of how good the customer service is and how customer friendly the business is.</p><blockquote>Your services must be observable.</blockquote><h4>Are you able to move to a new more suitable technology with relative ease?</h4><blockquote>This test reveals how coupled your platform is to a technology and thus determines your ability to modify and adapt to changing technology landscape.</blockquote><p>Apart from some rare scenarios (think old school AS400) inability to change programming language or implementation technology smells of bad design.</p><p>This can happen:</p><ul><li>because of implementation strategies such as “library sharing” (JARs, DLLs etc) for critical cross functional logic</li><li>because of over reliance on a particular operating system by directly embedding system level commands in the code.</li></ul><p>This can obviously be very damaging from a maintainability stand point which in turn can adversely impact cost. But the large impact is on the ability to pivot and innovate fast. the COVID-19 Pandemic tested the ability of businesses to adapt to an ever changing situation and innovate on the fly.</p><blockquote>A platform that can adapt and change by design is highly desirable and can open up new opportunities for business with relative ease.</blockquote><h3>Summary</h3><p>Micro services have seen wide adoption in the technology landscape, with all of their touted benefits. It is essential though to understand that they are not always the best architectural choice for a business.</p><p>Implemented correctly they can yield significant benefits in terms of reliability, scalability and cost and pave the way for agility to be put in practice. Equally, with bad implementation practices micro services can be worse than monoliths from a maintainability &amp; reliability perspective.</p><p>The <em>litmus tests</em> are a starting point that if applied consistently and continually to your micro services can ensure that you do not veer off track and create maintainable micro services.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4009002a9018" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Storing Database Credentials Securely]]></title>
            <link>https://medium.com/@siddhesh.jog/storing-database-credentials-securely-4bab8165d798?source=rss-95a776206365------2</link>
            <guid isPermaLink="false">https://medium.com/p/4bab8165d798</guid>
            <category><![CDATA[aws-secrets-manager]]></category>
            <category><![CDATA[manager-secrets]]></category>
            <category><![CDATA[credentials]]></category>
            <category><![CDATA[database-security]]></category>
            <category><![CDATA[access-control]]></category>
            <dc:creator><![CDATA[siddhesh jog]]></dc:creator>
            <pubDate>Fri, 03 Apr 2020 14:13:07 GMT</pubDate>
            <atom:updated>2020-04-03T14:13:07.809Z</atom:updated>
            <content:encoded><![CDATA[<p>Securing access to databases requires setting up a multi-layered perimeter:</p><p><strong><em>Network security</em></strong> to ensure that the database can only be <strong><em>reached </em></strong>from certain networks. This can be achieved by creating private restricted networks and hosting databases on them and granting access to those networks only on a need basis.</p><p><strong><em>Physical security</em></strong> to ensure that those that breach the network perimeter and have <strong><em>access to raw data</em></strong> can still not make heads or tails of it. Encrypting data at rest, obfuscation, federation, etc can help in this regard.</p><p><strong><em>Access control </em></strong>to only allow access to the database with valid credentials.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vABIBy1qO-WdgLB8LxU_6w.jpeg" /><figcaption>Keep your credentials under lock and key</figcaption></figure><blockquote>Access control not only guards against wholly unauthorized access but also allows for the creation of fine-grained access driven authorization levels.</blockquote><p>It is therefore essential that access credentials be stored securely to guard against unauthorized access. Here we will look at 5 ways to store access credentials and when to use them.</p><h3>1. Hardcoded in Configuration File</h3><p>The idea is to use a properties file or framework-specific configuration file to store credentials and then use it in code at run time. While it is extremely easy to get going; the credentials will make their way into the version control system and thus extremely vulnerable to theft and misuse.</p><p><strong>Pros</strong></p><ul><li>Zero implementation time.</li></ul><p><strong>Cons</strong></p><ul><li>Credentials available to anyone with access to the code repository.</li><li>Changing credentials will need a code change, <strong><em>which becomes infinitely worse in a multi-module platform.</em></strong></li></ul><h4>When to use</h4><ul><li>Proof of concept stage when a test database is being used.</li><li>Development with a local database.</li></ul><h3>2. Place holders in the Configuration file</h3><p>This is a significant improvement over the previous approach, from a security standpoint. Place holders can take the form of <em>$Username, $Password. </em>These parameters can then be replaced during the build and packaging process.</p><p>The credentials can be stored as environment variables in the CI tool with only admin having access to these.</p><p><strong>Pros</strong></p><ul><li>Credentials not stored in code and hence not available in the repository.</li><li>Changing credentials does not require an extensive code change across all modules.</li></ul><p><strong>Cons</strong></p><ul><li>Credentials stored in Jenkins are accessible to Jenkins users.</li><li>Since the password is replaced in the config file it may still be visible on the server.</li></ul><h4>When to use</h4><ul><li>Early stages of a greenfield project with few modules.</li></ul><h3>3. Store in Secret Manager</h3><p>This approach builds on the previous one by storing the secret in a <em>secret manager </em>such as <a href="https://www.vaultproject.io/docs/what-is-vault/">Hashicorp Vault</a>, <a href="https://square.github.io/keywhiz/">Keywhiz</a>, etc. A secret manager does a lot more than merely store secrets. It can audit access, dynamically rotate credentials, encrypt secrets at rest.</p><p>Such secrets can be accessed by the CI pipelines by employing a token mechanism.</p><blockquote>This introduces a secondary security problem of storing the access key to the secret manager secretly.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/566/1*t0kNEHQ1kPPvCOWSsBxCGw.jpeg" /><figcaption>How should I secure access to vault??</figcaption></figure><p><strong>Pros</strong></p><ul><li>A secrets manager is purpose-built for securely storing secrets and comes with features such as encryption at rest &amp; fine-grained access control, credential rotation.</li></ul><p><strong>Cons</strong></p><ul><li>The secret manager password/credentials/keys still need to be stored securely. While the database password is secure in a vault, the vault password needs to be secured.</li></ul><h4>When to use</h4><p>This is an excellent choice for multi-module/microservices production deployments.</p><h3>4. Programmatic in-memory storage</h3><p>While storing password place holders in config files is a good start; creating a data source and connection pool programmatically will ensure that the database credentials are not stored on the build server (after parameter replacement in code) or on on the drive on the production server and are thus impervious to theft.</p><p>Secret managers (such as Vault) come with a simple HTTP API and hence can be directly accessed from code.</p><pre><em>/**<br> * Create datasource programatically.<br> */<br></em>@Bean<br>public DataSource getDataSource(){<br>    <br>    DataSourceBuilder dataSourceBuilder = DataSourceBuilder.<em>create</em>();<br>    dataSourceBuilder.type(org.apache.tomcat.jdbc.pool.DataSource.class);<br>    PoolProperties p = new PoolProperties();<br>    p.setUrl(config.getString(&quot;url&quot;));<br>    p.setDriverClassName(&quot;com.mysql.cj.jdbc.Driver&quot;);<br>    p.setUsername(config.getString(&quot;username&quot;));<br>    p.setPassword(config.getString(&quot;password&quot;));<br>    p.setTestWhileIdle(false);<br>    p.setTestOnBorrow(true);<br>    p.setValidationQuery(&quot;SELECT 1&quot;);<br>    p.setMaxActive(config.getInt(&quot;maxActive&quot;));<br>    p.setMaxIdle(config.getInt(&quot;maxIdle&quot;));<br>    p.setMinIdle(config.getInt(&quot;minIdle&quot;));<br>    p.setMaxWait(config.getInt(&quot;maxWait&quot;));<br>    p.setInitialSize(config.getInt(&quot;initialSize&quot;));<br>    DataSource dataSource = dataSourceBuilder.build();<br>    ((org.apache.tomcat.jdbc.pool.DataSource)dataSource).setPoolProperties(p);</pre><pre>return dataSource;<br>}</pre><p><strong>Pros</strong></p><ul><li>Credentials not stored on the hard drive and hence much more secure.</li></ul><p><strong>Cons</strong></p><ul><li>The vault credentials have to be stored in the code or config (or as a Jenkins parameter). This can become a security issue.</li><li>Coding required for otherwise configurable components like data source &amp; connection pools.</li></ul><h4>When to use</h4><p>This technique can be used in production but can be tricky with microservices architecture. You will be required to create a data source provider module that creates the data source &amp; connection pool. Such a module can then be embedded in other modules to avoid code repetition.</p><h3>5. Cloud-Native Secret Manager</h3><p>A cloud-native secret manager, like AWS’ <strong><em>Secrets Manager, </em></strong>combined with programmatically created data sources is the most secure way of storing credentials.</p><p>In AWS permissions to access resources, like the secrets manager, can be granted to EC2 instances employing IAM Policies and Roles. Thus no credentials are required to access Secrets manager.</p><p>This approach thus has all the benefits of employing the secret manager and programmatic data source, and an added benefit of not needing credentials to access the secret manager.</p><pre><strong>/* Sample Java code for fetching secrets from secret manager. AWS provides this in javascript, go, java, ruby, python, C# */</strong><br>String region = &quot;ap-southeast-2&quot;;</pre><pre>// Create a Secrets Manager client<br>AWSSecretsManager client;</pre><pre><em>log</em>.info(&quot;SecretName: {}&quot;, secretName);<br><em>log</em>.info(&quot;useCreds: {}&quot;, useCreds);</pre><pre>client  = AWSSecretsManagerClientBuilder.<em>standard</em>()<br>        .withRegion(region)<br>        .build();</pre><pre>//get Secret from secrets manager.<br>String secret, decodedBinarySecret;<br>GetSecretValueRequest getSecretValueRequest = new GetSecretValueRequest()<br>        .withSecretId(secretName);<br>GetSecretValueResult getSecretValueResult = null;<br><strong>//sample code, do not handle exception like this</strong><br>try {<br>    getSecretValueResult = client.getSecretValue(getSecretValueRequest);<br>} catch (Exception e) {<br>    throw new RuntimeException(e);<br>}</pre><pre><em>log</em>.info(&quot;*************************************&quot;);<br><em>log</em>.info(&quot;Loading data from AWS SM&quot;);</pre><pre>JSONObject object = null;</pre><pre>// Decrypts secret using the associated KMS CMK.<br>// Depending on whether the secret is a string or binary, one of these fields will be populated.<br>if (getSecretValueResult.getSecretString() != null) {<br>    secret = getSecretValueResult.getSecretString();</pre><pre>object = new JSONObject(secret);<br>}<br>else {<br>    decodedBinarySecret = new String(Base64.<em>getDecoder</em>().decode(getSecretValueResult.getSecretBinary()).array());<br>    object = new JSONObject(decodedBinarySecret);</pre><pre>}</pre><pre><em>log</em>.info(&quot;*************************************&quot;);</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/576/1*YbA-bU1ftwBxqG1boKWBQw.jpeg" /><figcaption>Datasource created programmatically</figcaption></figure><p><strong>Pros</strong></p><ul><li>No credentials required to access Secrets manager, cascading secret protection problem solved.</li><li>An inexpensive method for storing secrets with powerful features such as RDS credential rotation and lambda hooks.</li></ul><p><strong>Cons</strong></p><ul><li>Larger dependency on cloud-native features would mean cloud migration is that much more difficult.</li></ul><h3>Summary</h3><p>Some of the above approaches are better than others. Yet your context will drive the solution that you adapt.</p><p>At the very least use a <strong><em>secret manager</em></strong>. You may or may not employ the programmatic approach based on your choice of technology and how easy or difficult it is to implement this approach.</p><p>And if you can write modular build pipelines and component generation code then cloud-native is the way to go.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4bab8165d798" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The road to AWS Cost Optimization]]></title>
            <link>https://medium.com/@siddhesh.jog/the-road-to-aws-cost-optimization-8b7e2f852b89?source=rss-95a776206365------2</link>
            <guid isPermaLink="false">https://medium.com/p/8b7e2f852b89</guid>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[cost-optimization]]></category>
            <category><![CDATA[performance-testing]]></category>
            <dc:creator><![CDATA[siddhesh jog]]></dc:creator>
            <pubDate>Sat, 15 Feb 2020 16:06:48 GMT</pubDate>
            <atom:updated>2020-02-15T16:06:48.075Z</atom:updated>
            <content:encoded><![CDATA[<p>Building and maintaining software platforms in a fast-paced startup environment can be very tricky. The turn around time expectations are always sky high, product/feature releases are expected <em>yesterday</em>. As far as the technology platform is concerned there is a tendency to cut corners across the board: be it in module design; component reusability, architecture or infrastructure.</p><blockquote>Provisioning infrastructure in the cloud is incredibly easy. A few clicks and you have a group of servers, load-balancer, firewall; routing rules, etc.</blockquote><p>Our experience at Parcelpoint was no different. With major integration projects and new product versions rolling out thick and fast; and a <em>“click-start” </em>cloud platform (AWS) at our disposal, it did not take long for infrastructure size and cost ballooned up.</p><p>This was a major talking point in the 2019 platform review conducted by the tech team. It was not just the cost but also the sheer spread of the infrastructure that was concerning.</p><h3>Testing to understand our infrastructure need</h3><p>We began by asking ourselves, <strong>“<em>How could we optimize our infrastructure?”</em></strong> and quickly moved to <strong>“<em>What exactly constitutes an optimal infrastructure as far as our platform is concerned?”</em></strong></p><p>There simply is no way around it. Non-functional requirements (NFRs) get neglected more often than not; especially when building systems at pace; and without them, all you can do is make guesstimates regarding your infrastructure needs.</p><p>In the last couple of years, we have been migrating a fairly monolithic platform into an SOA (i shudder to say microservices-based) based platform while delivering new product features.</p><blockquote>Cloudformation templates &amp; AWS CLI made life very (too) easy as far as provisioning/modifying infrastructure was concerned but considerations about optimal needs (were sidetracked) were not front and center.</blockquote><p>As such the easy option was to go with the general-purpose production-grade servers (m3/m4/m5s) and club services and containers together based on a guesstimated tiering system.</p><p>Oct 2019 is when we changed all that and embarked on detailed performance testing that took into <strong><em>consideration real-world scenarios</em></strong> and projected growth rate (with a buffer of course).</p><blockquote>It’s critical to conduct performance testing rooted in the realms of reality.</blockquote><p>We looked at data from the past 4 years to figure out the time of day when load spikes occurred.</p><h4>Load Modelling</h4><p>We ensured that we were <strong>NOT</strong> planning infrastructure for:</p><ul><li>Excessive (ex: 200 times) increase in usage: This is a DDoS scenario that is not for the compute infrastructure to handle.</li><li>Infrequent Spikes and peaks; <strong><em>something auto-scaling could handle.</em></strong></li></ul><blockquote>It is important to understand that auto-scaling has been misunderstood to be a long term scalability enabler when in fact it is an elasticity enabler that allows you to stretch infrastructure to meet infrequent/known spikes and peaks.</blockquote><p>We used existing data and usage patterns to extrapolate infrastructure needs for x fold increase in load (<strong><em>x</em></strong><em> is for me to know</em>).</p><h4>Monitoring performance</h4><p>It is one thing to model loads and loads the system quite another to accurately monitor and understand the usage patterns. <a href="https://www.dynatrace.com/">Dynatrace</a>, our monitoring tool, provided detailed insights about how our applications and services were performing under load, allowing us to better group containers/services.</p><p>We were able to gather statistics about response times, CPU usage, memory usage; with a detailed breakdown of memory utilization by the software component.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*A8i2XgHuI35ENYLPV-clEw.png" /><figcaption>response time statistics</figcaption></figure><h4>Testing to the breaking point</h4><p>By mid-November, we knew what our infrastructure scalability needs were and were pleasantly surprised by just how much we could optimize.</p><p>To figure out our elasticity needs, we tested the applications and APIs to breaking point allowing us to better configure our autoscaling groups in AWS.</p><h4>Fixing, that which should be fixed</h4><p>Performance testing revealed certain issues that were easy enough to fix and would bring big benefits; especially poorly written queries or ill-managed OOPs code structure that lead to major memory burn.</p><p>Although it meant increasing/changing the scope of what was a cost optimization exercise, we could not let this go and ended up making these code changes.</p><h3>Using AWS Tools for insights and guidance.</h3><p>Having figured our needs our next task was to review and verify our findings by a secondary means.</p><p>This is where the AWS service kit shone brightly. AWS Trusted Advisor is a tool that guides to help you provision your resources following AWS best practices</p><h4>AWS Trusted Advisor</h4><p>Trusted Advisor is just phenomenal. On the cost optimization front its checks for a whole host of things; like low utilization of ec2 instances, underutilized volumes; idle DB instances.</p><p>It, of course, does a lot more than that and provides performance, security, fault tolerance checks as well as information about service limits. Check it out <a href="https://aws.amazon.com/premiumsupport/technology/trusted-advisor/">here</a>.</p><p>Trusted advisor allowed us to cross-reference our performance tests findings, and pretty much gave us huge confidence that we were on the right track.</p><h4>AWS Compute Optimizer</h4><p>AWS Compute Optimizer recommends optimal AWS Compute resources for your workloads to reduce costs and improve performance by using machine learning to analyze historical utilization metrics.</p><p>While performance tests and Trusted Advisor, told us where we were underutilizing compute resources; <strong><em>Compute Optimizer gave specific targeted advice on rightsizing instances</em></strong>. This was perfect. It meant we did not have to waste time fiddling around different instance types and families on our downsizing journey.</p><h4>AWS Pricing Calculator</h4><p>The Pricing Calculator is a neat tool. It allows you to create an estimate of your architecture; in our case the entire platform. It allowed us to compare the cost of our “<strong><em>On Demand</em></strong>” compute instances with the cost if we reserved these instances for 1/3 years. This was great because it allowed us to <strong><em>“see”</em></strong> the benefits before they materialized.</p><h3>Modify, Commit and Save</h3><p>All that was left now was the modify the infrastructure. This <strong><em>was/is being </em></strong>achieved by:</p><ul><li>Re-grouping and re-distributing services across instances.</li><li>Modifying (mostly downsizing) instances to meet present and future performance demands.</li><li>Reserving instances for EC2 as well as RDS to bring down compute costs.</li><li>Deploying services on containers to allow for disparate services to be grouped into a single instance thus further improving utilization and reducing cost.</li></ul><blockquote>It has been a rather grinding but fruitful journey of infrastructure optimization and it has thus far yielded this:</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Hq7qpfbu-4UDuXhVDIz9bg.png" /><figcaption>cha-ching*</figcaption></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8b7e2f852b89" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[An SQL Based rule engine]]></title>
            <link>https://medium.com/@siddhesh.jog/an-sql-based-rule-engine-6a187e2d88d8?source=rss-95a776206365------2</link>
            <guid isPermaLink="false">https://medium.com/p/6a187e2d88d8</guid>
            <category><![CDATA[rule-engine]]></category>
            <category><![CDATA[database]]></category>
            <category><![CDATA[sql]]></category>
            <dc:creator><![CDATA[siddhesh jog]]></dc:creator>
            <pubDate>Thu, 24 Oct 2019 16:23:19 GMT</pubDate>
            <atom:updated>2019-10-29T23:01:50.606Z</atom:updated>
            <content:encoded><![CDATA[<p>Over the past year, we migrated our monolithic platform into a more component-based one. We were able to modularize the platform to the extent that logical service functions are now performed by individual API modules.</p><p>Thus there are individual modules to perform notification, label generation, etc. The old code had huge if-else code blocks for fine-tuning notifications (email/SMS), label generation, lifecycle tracking, etc.</p><p>When we migrated to new small individual modules we wanted to ditch these if-else blocks for something that was</p><ul><li>maintainable: easy to read, easy to modify and incrementally test</li><li>performant: would execute quickly without straining underlying infrastructure resources.</li><li>flexible: could be used across multiple modules without significant change.</li><li>cost-efficient: not be a burden on our wallet.</li></ul><p>With cost being a factor we first assessed Drools. While it was straight forward to write rules with Drools, it fell flat on the maintainability (<em>it was/is difficult to test the rule engine execution) </em>and performance (<em>execution of rules often required large amounts of data fetched from the database and was slow) </em>fronts.</p><p>Having broken down the platform into individual components, we could embed rules engines in each component and in that regard Drools rule engine, was flexible and of course quite cost-efficient.</p><p>We have since replaced it with an in-house SQL based rule engine.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lbeata4yK0nRayfbjA73EQ.png" /><figcaption>Rules!</figcaption></figure><p>It essentially has three parts:</p><p><strong>1)</strong> A database configuration table <strong><em>rule_config </em></strong>that holds all the rules with the following key fields:</p><ul><li>rule_module: all related rules belong to the same module: example “SMS Rules”</li><li>rule_sql: the SQL that represents the rule and always ends with entity_id = :v_entity_id,</li><li>rule_description: Human-readable description of the rule so that we understand what the rule is for.</li><li>sort_order: to sort the rules.</li><li>rule_action_flag: to determine if this is a negation rule. Should the action be taken or not if the rule is a hit. For example, a rule for NOT sending an email.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/216/1*ZRfuTJtMrxioQFF8_XN85A.png" /><figcaption>rule configuration table.</figcaption></figure><p>The rule_sql always takes the following form:</p><pre><strong>SELECT</strong> count(e.id) <br>  <strong>FROM</strong> entity e <br>  <strong>JOIN</strong> &lt;some_joins_if_needed&gt; <br> <strong>WHERE</strong> &lt;some_conditions&gt; <br>   <strong>AND</strong> e.entity_id = :v_entity_id</pre><p>When the rule SQL is executed for a particular entity_id its response is 0, if the rule does not hold good for that particular entity and &gt; 0 if it does.</p><p>For example: consider a rule that selects all employees in the Math department</p><pre><strong>SELECT</strong> count(e.id) <br>  <strong>FROM</strong> employee e <br>  <strong>JOIN</strong> employee_department ed <strong>ON </strong>e.id = ed.employee_id   <br>  <strong>JOIN</strong> department d <strong>ON </strong>d.id = ed.department_id and d.name = &#39;Math&#39;<br> <strong>WHERE</strong> e.id = :v_entity_id</pre><p>This rule was return 1 for all employees that belong to the department Math and 0 for those that do not. This rule could thus be used for sending tailored emails to employees of the Math department.</p><p><strong>2)</strong> A stored procedure <strong>execute_rules, </strong>that executes the rule SQLs for a particular module for the given entity_id and returns the rule_id of the rule_id for that rule that got the <em>“hit”.</em></p><pre><strong>execute_rules</strong>(<strong>IN</strong> <em>p_rule_module</em>, <strong>IN</strong> <em>p_entity_id</em>, <strong>OUT</strong> <em>p_rule_id</em>)</pre><p>The procedure essentially first fetches all rule_config rows for a particular rule_module with <strong><em>ascending</em></strong> sort_order.</p><p>It then replaces the, “:v_entity_id” parameter with the specific entity id of the rule and fires the query.</p><pre><strong>SET</strong> v_sql = v_rule_config.rule_sql;</pre><pre><strong>SET</strong> @SQL = <strong>REPLACE</strong>(v_sql, &quot;:v_entity_id&quot;, p_entity_id);</pre><pre><strong>PREPARE</strong> v_sql <strong>FROM</strong> @SQL;</pre><pre>EXECUTE v_sql;</pre><p>If the resultant count is &gt; 0 then the rule has been hit and rule_id is returned from the stored procedure.</p><p>The stored procedure is a <strong>MUST</strong>. The queries themselves are quite inexpensive (if written correctly). Using JPA/Hibernate to execute each query though would lead to way too many I/O operations with the associated serialization-deserialization (This was still faster than Drools). The stored procedure allows for 1 journey to the database.</p><p><strong>3)</strong> The third bit is a standalone spring java jar module with a <strong><em>RuleEngineService</em></strong> that calls the stored procedure and then fetches the entire <strong><em>RuleConfig </em></strong>object once the procedure returns the rule_id.</p><pre>public class RuleEngineService {</pre><pre>   public <em>RuleConfig</em> <strong>fireRules</strong>(String ruleModule, String parcelId){<br>      <em>log</em>.info(&quot;Firing rules for module {}&quot;, ruleModule);<br>      Long ruleId = <em>ruleExecutor</em>.executeRules(ruleModule, parcelId);<br>      <strong><em>return</em></strong> <em>ruleConfigRepository</em>.findById(ruleId).get();<br>   }</pre><pre>}</pre><p>This jar is then added to individual services to embed the rule engine into the service.</p><p>The key though is that the rule engine only makes a decision. It does not execute the action. That is left to the rule invoking module.</p><p>Right off the bat, it feels like a step back in time. The stored procedure locks you in with the RDBMS as well as a specific DB engine.</p><p>But it has quite a few advantages and delivers on most of the objectives we had to start with:</p><ul><li>Maintainability: It is quite easy to maintain rules, modify them and test them, with basic JUnit testing in the standalone rule jar for each module. <strong><em>It does need half decent query writing skills and needs technical skills to write new rules though.</em></strong></li><li>Performance: There has been a major upgrade from a performance standpoint, <strong><em>more than halving the response time</em></strong> of the APIs where it has been used (as compared to drools). The stored procedure executes all SQLs in one go thus reducing the time and resources used for serialization-deserialization. <strong><em>We did try to hibernate to executes SQLs one by one from the java module but the time gain when using stored procedure was outstanding</em></strong>. Large data sets were fetched into the service for the Drools engine to make a decision. That is no longer required. It has allowed us to trim down the domain model inside the APIs/services; no need to fetch and store fields that are required to make the decision. <strong><em>Only fetch and store data required to provide the service</em></strong> <em>(notification/label configuration etc). </em>The database load has also gone down; with small queries dependant on IOPS rather than throughput.</li><li>Flexibility: easy to modify/add rules (a bit too easy); as all you need to do is alter the SQL. There is no sprint development time required. <strong><em>This was particularly problematic with Drools as changing rules required sprint development.</em></strong></li></ul><p>On the flip side:</p><ul><li>Business team members (and even tech ) without SQL knowledge cannot update rules.</li><li>SQL Rules are susceptible to the introduction of bugs and need a strict regimen and process for modifying/adding rules.</li><li>You are effectively tied to the database. It would not be hard to change from MySQL to Oracle/Postgres etc. But changing to a NoSQL database is now harder.</li><li>If there are 100s of rules within a module, the engine may run into performance.</li></ul><p>To summarize; The SQL rule engine is a great fit for use cases where rules run its 10s rather than in the 100s or 1000s. It has heavy tech dependency but is quite maintainable; super flexible and performant.</p><p>It has met the goals that it was built for and has allowed us to rapidly change/add rules and significantly improve the performance of rule-based services.</p><p>Code: <a href="https://github.com/tech-parcelpoint/rule-engine">https://github.com/tech-parcelpoint/rule-engine</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6a187e2d88d8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Monolithic architecture: 10 tips to take it on.]]></title>
            <link>https://medium.com/@siddhesh.jog/monolithic-architecture-10-tips-to-take-it-on-1c6e64bba1ac?source=rss-95a776206365------2</link>
            <guid isPermaLink="false">https://medium.com/p/1c6e64bba1ac</guid>
            <category><![CDATA[architectural-design]]></category>
            <category><![CDATA[microservices]]></category>
            <category><![CDATA[transformation]]></category>
            <dc:creator><![CDATA[siddhesh jog]]></dc:creator>
            <pubDate>Wed, 06 Mar 2019 13:45:37 GMT</pubDate>
            <atom:updated>2019-03-07T00:48:05.460Z</atom:updated>
            <content:encoded><![CDATA[<p>In <a href="https://medium.com/@siddhesh.jog/monolithic-architecture-understanding-why-it-came-to-be-dc8d9ac5878c">Part 1</a> we tried to figure out the reasons behind the organic development of a monolithic infrastructure. Now that you find yourself facing the myriad problems associated with the monolithic architecture (unnecessary deployments due to dependencies, QA uncertainty about what may break, unpredictable bugs caused by seemingly unassociated code etc) how do you go about getting out of this quagmire?</p><p>It’s a grind. It is going to take all your design skills backed by skillful development and precise planning.</p><p>In attempting to perform this transformation I faced a number of issues and roadblocks and made a few errors in judgment. Learning from those, am going to share with you 9 tips to help you in the process of transforming your monolithic platform to a component-based one.</p><ul><li><strong><em>One: </em></strong>Before you do anything else, <strong><em>identify the components in your platform</em></strong><em>. These could be functional service components (based on verbs) or RESTful resources.</em> For the cinema house platform, these could be ticket generator (or ticket resource), notification (or notify service) etc. Every service may be represented as a resource or an SOA service. Thus notification/ticket may serve as restful resources and notify, generate-ticket may server as SOA endpoints. At this stage, all you need to do is identify uniquely deployable components.</li><li><strong><em>Two:</em></strong> <strong><em>Start with the easiest of the components</em></strong> discovered in step 1. Remember not only will your team need to develop, test and deploy a new component and integrate it with the existing platform, but also they will be required to do the foundational work i.e create a common response wrapper for your services, finalize exception handling mechanism, modify test strategy and test setup and create independent component deployment scripts. These activities will be challenging the first time around; hence start with the easiest component.</li><li><strong><em>Three: </em></strong>Design the component so it can be <strong><em>plugged into the existing framework and toggled on-off with minimal configuration changes</em></strong>. This will simplify the points of integration and make it considerably easy to test and “be confident” about the change. Ex: In the case of the ticket generating code; once the service is created and deployed create a Generator Factory in the monolithic website application and based on certain configuration pass a small percentage of the ticket requests to the new service; slowly ramping up the usage of the new service as performance data comes through and functional bugs are ironed out.</li><li><strong><em>Four: Build a common logging component/process/practice</em></strong>. Ensure that services are designed to share the log identifier for requests that may be daisy chained at times (DO NOT daisy chain requests as far as possible).</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/711/1*1_8jW2gdr-5I2IRtHESA4A.jpeg" /></figure><ul><li><strong><em>Five:</em></strong> Remember <strong><em>one size does not fit all.</em></strong> Do not be stubborn. Weigh out your options. Don’t use only REST methodology for your services, or rely only on the Spring framework as the framework of choice. While it is essential to not get carried away with the tech stack; sometimes its best to tailor solutions. A highly cached service, for example, “may” be best built as a serverless service (ex: AWS Lambda) that sits behind a CDN. Yet others that need mathematical computation may be best built using spring boot and a suitable rule engine.</li><li><strong><em>Six: Tailor your infrastructure to code</em></strong>. In our example, the ticket generator may need loads of memory in order to store the ticket(s) in memory during the generation process, the notification service may need a general purpose machine, a seat availability service may require a high CPU machine, a service built expressly for serving front end may be built on a lightweight system if it employs node+SPA. (In AWS: Use tags to identify services, environments, and infrastructure so that CLI scripts may be used to bring up/tear down infrastructure)</li><li><strong><em>Seven: Offline anything and everything that need not be online</em></strong>. Do not make your consumer wait when they are not required to do so; do not add to the OLTP time by including service calls that may be carried out by downstream components that are invoked asynchronously. A booking email is a perfect example of one such task that need not be done synchronously.</li><li><strong><em>Eight: Abstract communication tech from the components. </em></strong>Building numerous smaller components automatically implies increased inter-component communication requirements. Do not expose your components to communication tech (example: MQ, CICS etc). Use ESB to wire the components together using HTTP. This will allow you to change providers (ex: Active MQ to SQS), versions without having to modify individual components and reduce testing requirements inter-component connectivity</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1011/1*pP1JuTCF0nZNajhezOuTJQ.jpeg" /></figure><ul><li><strong><em>Nine: Rely on the SOLID principles</em></strong>. From an architectural standpoint especially on S. Separation of concerns is key; its the reason you started with this transformation in the first place. Ensure that each component remains independently testable (thus derisking deployments ). This is the crux of the transformation project; without it you have no transformation at all, just smaller monoliths.</li><li><strong><em>Ten: Absolutely 100% avoid jar sharing as far as possible. </em></strong>(Yeah OSGI you…no no). Using a jar with common classes across components may seem enticing but it is a dangerous trap. Sooner or later some developer will put functionality in the jar that does not belong there. A JAR is designed to deliver “infrequently changing utility” function. Do not put domain objects common functional services and share them across components. On the other hand, all validations may be put in one module and shared by mean of a JAR.</li></ul><p>Do all of the above and bring your best self to the table day in day out and, you will get the architectural transformation that you desire.</p><p>Of course, there will be other tips that may help that you know of and I would love to hear about those as well. So do put those in the comments.</p><p>Oh and one last thing; this will also be the most fun thing that you take on. So do not forget to enjoy it.</p><p>All the best.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1c6e64bba1ac" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Monolithic Architecture: Understanding why it came to be.]]></title>
            <link>https://medium.com/@siddhesh.jog/monolithic-architecture-understanding-why-it-came-to-be-dc8d9ac5878c?source=rss-95a776206365------2</link>
            <guid isPermaLink="false">https://medium.com/p/dc8d9ac5878c</guid>
            <category><![CDATA[architecture]]></category>
            <category><![CDATA[microservices]]></category>
            <category><![CDATA[monolithic-architecture]]></category>
            <dc:creator><![CDATA[siddhesh jog]]></dc:creator>
            <pubDate>Wed, 06 Mar 2019 13:44:18 GMT</pubDate>
            <atom:updated>2019-03-06T23:15:11.060Z</atom:updated>
            <content:encoded><![CDATA[<p>Most architecture is “Not bad” in the beginning.</p><p>It’s like Harvey Dent says in “The Dark Knight”: “You either get transformed to a new paradigm/methodology or stay around long enough to become rotten architecture”.</p><p>Monolithic architecture is one such relic. It is definitely not architecture without components, but rather it is an architecture with vertical/almost functional components.</p><p>If you were to look at platform code written around 2006–13 (or sometimes even more recent) you would more often than not see the following:</p><ol><li>A website for consumers.</li><li>A set of APIs/Services for integration partners</li><li>Batch jobs to perform batch tasks.</li><li>Administration portal for back-end admins/user admins.</li><li>A portal for business partners/customers.</li><li>All components updating the data in the database.</li><li>Huge functionality/CRUD replication across components.</li></ol><p>As is evident from the list the components are very functional. with business needs transformed into almost “<em>to the letter</em>” components.</p><p>Why where these components built in such a manner? While there are a number of factors causing this the most pertinent are a time crunch, lack of an architectural vision and lack of domain knowledge.</p><p>Let&#39;s take a look at a cinema house IT platform where something like the following transpires.</p><ul><li>“We need a website where customers <em>can book tickets</em>” → MVC based spring website with ticket booking, printing, notification etc.</li><li>“We need a way for customer service executives to help customers <em>book tickets; </em>you know! when they are called over the phone!” → A website for customer service reps.</li><li>“X booking website is introducing a new feature and want to<em> sell our tickets</em> through their front end; they have great market reach; this is a great opportunity” → ticket booking API for integration partner.</li><li>“We need to send out ticket availability information to our partners every day EOD, they won’t work with APIs, and we need this partnership” → Batch job to provide information about ticket availability.</li><li>“We need to send ticket availability information to partner X; they want to display it on their website; LIVE” → API for ticket availability</li></ul><p>and so on…</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/577/1*8CjIArpGE3IoTSHhMS-Fig.jpeg" /><figcaption>Expedient coding over prudent design</figcaption></figure><p>It all feels like a business need (<em>read</em> <strong><em>requirements</em></strong>) problem rather than an architectural/solution problem. The speed of developing new functionality and insufficient knowledge of what is to come can be architecturally crippling.</p><p>In the case above; the cinema house was probably first to adapt to tech and build their own booking service. But as the online landscape evolved centralized booking websites not only sprang up but also became quite successful and necessary for business. the cinema house needed to expose a web service to do what its website was already doing: book tickets.</p><p>What they are left with is significant code replication across components. A change in the booking logic, new ticket look, and feel; a new notification requires deployment across the platform. Moreover, each of the smaller sub-modules will get complex over time with rules tailored to meet specific partner’s expectation adding to the maintainability problem.</p><p>If a change was required in the notification code this would need at the very least a full retest of all the components and sometimes a full system-wide redeployment. A domain model change would absolutely necessitate a full-scale redeployment after thorough regression testing.</p><p>As is evident, once the system matures the change to effort ratio can get quite skewed. In a highly coupled platform such as this, it will be next to impossible for QA professionals to certify a change by just testing the change, because of the fear that “something somewhere will break if we change <em>this </em>part”</p><p>Business needs are driven by changing market realities. Architecture “<strong><em>must”</em></strong> cope with changing business needs, and while there can never be an architecture that never needs to be changed; we can make life easy for ourselves by building a component-based architecture that is more adaptable to change.</p><p>In <a href="https://medium.com/@siddhesh.jog/monolithic-architecture-10-tips-to-take-it-on-1c6e64bba1ac">part 2 </a>we will look at how to transform a functional architecture such as this into a component based one and the guiding principles for building such an architecture ground up.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dc8d9ac5878c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Exposing a Web Service using JAX-WS]]></title>
            <link>https://medium.com/@siddhesh.jog/exposing-a-web-service-using-jax-ws-e65dea006d4e?source=rss-95a776206365------2</link>
            <guid isPermaLink="false">https://medium.com/p/e65dea006d4e</guid>
            <category><![CDATA[java]]></category>
            <category><![CDATA[web-services]]></category>
            <category><![CDATA[jax-ws]]></category>
            <dc:creator><![CDATA[siddhesh jog]]></dc:creator>
            <pubDate>Tue, 23 Oct 2018 12:15:26 GMT</pubDate>
            <atom:updated>2018-10-23T12:15:57.717Z</atom:updated>
            <content:encoded><![CDATA[<p>We are going to look at a short (and completely useless from a production standpoint) example of exposing a web service using JAX-WS.<br> <br> We will create a simple method to get a Single Employee data. You can then build from there to further enhance the functionality.<br> <br> We are not going to look into deploying this web service on a web server. That is for a separate post. <br> <br> Lets Dive in:<br> <br> We will first create a simple maven project in eclipse: ExampleWSService</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/254/0*SaTryZ5CIyupCt8T.jpg" /></figure><p>First add your maven dependency:</p><pre>&lt;dependencies&gt; &lt;dependency&gt; &lt;groupId&gt;javax.xml.ws&lt;/groupId&gt; &lt;artifactId&gt;jaxws-api&lt;/artifactId&gt; &lt;version&gt;2.2.6&lt;/version&gt; &lt;/dependency&gt; &lt;/dependencies&gt;</pre><p>Create a new interface EXWSService which will serve as our SEI (service endpoint interface). This is not a a necessary step but it is always good to specify your exposed methods in an interface and then have an implementation class for the business logic.</p><p>import javax.jws.WebMethod;<br> import javax.jws.WebResult;<br> import javax.jws.WebService;<br> import javax.jws.soap.SOAPBinding;<br> import javax.jws.soap.SOAPBinding.Style;<br> import javax.jws.soap.SOAPBinding.Use;<br> import javax.xml.ws.ResponseWrapper;<br> <br> import com.study.ws.domain.Employee;<br> <br> @WebService(name = “EXWSService”)<br> @SOAPBinding(style = Style.DOCUMENT, use = Use.LITERAL)<br> public interface EXWSService {<br> <br> @WebMethod(operationName = “getEmployee”, action = “getEmployee”)<br> @ResponseWrapper(className = “com.study.ws.jaxws.GetEmployeeResponse”, partName = “WSEmployee”)<br> @WebResult(name = “employee”)<br> public Employee getEmployee();<br> <br> }<br> <br> <br> <br> Annotations:</p><ul><li>@Webservice: Allows us to provide a name element and designates this interface as an end point. The name provided here will appear in the WSDL as the name of the PortType.</li><li>@SoapBinding: This annotation allows us to specify the message style and encoding. Based on whether we use DOCUMENT or RPC style the interpretation of some of the other annotations changes. We can also select the parameterStyle, which if set to BARE means that the method parameter is the entire body of the message and there is no wrapper element.</li><li>@WebMethod: This annotation is required on each service method that we are going to expose. We can specify the operation name and soap action.</li><li>@ResponseWrapper: This is required if the style of SOAPBINDING is DOCUMENT. You may specify a fully qualified class name and a part name. If the style is RPC or if the parameterStyle is BARE along with style being DOCUMENT the wrapper need not be specified.</li><li>@WebResult: You can specify the partName, name and whether the result is to be pulled from header.</li></ul><p>We then provide an implementation class as follows:</p><p>import javax.jws.WebService;</p><p>import com.study.ws.domain.Employee;</p><p>@WebService(endpointInterface = “com.study.ws.EXWSService”, serviceName=”EXWSServiceImpl”, portName=”EXWSServiceport”)</p><p>public class EXWSServiceImpl implements EXWSService{</p><p>public Employee getEmployee() {</p><p>return new Employee(1L, “James”, “Bond”);</p><p>}</p><p>public EXWSServiceImpl() {</p><p>super();</p><p>}</p><p>}</p><p>Now compile your project. Go to the classes folder. In this folder (if you have used exactly the same names as in here) run the following command:</p><p>wsgen -cp . -keep -verbose com.study.ws.EXWSServiceImpl.</p><p>wsgen utility creates the request and response wrapper classes that are required with DOCUMENT style of SOAPBINDING. The @RequestWrapper and @ResponseWrapper allow you to specify the details of such generated cass. Even if these annotations are not provided default values are used and wrapper classes are generated.</p><p>You should see output as below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/400/0*FhQtydZZZS0U0b81.jpg" /></figure><p>Add the generated classes to your source. If you view them you will see that they just wrap your request and response data.</p><p>Next create a publisher as follows:</p><pre>package com.study.ws.publisher;</pre><p>import javax.xml.ws.Endpoint;</p><p>import com.study.ws.EXWSServiceImpl;</p><p>public class Publisher {</p><p>public static void main(String[] args) {</p><p>Endpoint.publish(“http://localhost:8090/EXWSServiceImpl&quot;, new EXWSServiceImpl());</p><p>}</p><p>}</p><p>Run it and type the following url in your browser:</p><p><a href="http://localhost:8090/EXWSServiceImpl">http://localhost:8090/EXWSServiceImpl</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/400/0*Fc2QF_eE0hbvAXre.jpg" /></figure><p>The web service is now published and you may explore the generated WSDL. To understand the WSDL and relate its segments to java annotations have a look at the following article:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/400/0*cDsKqgtuDzyV2L_z.jpg" /></figure><p><em>Originally published at </em><a href="https://sjtn.blogspot.com/2016/01/exposing-web-service-using-jax-ws.html"><em>sjtn.blogspot.com</em></a><em> on October 23, 2018.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e65dea006d4e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[6 ways to get the most out of your stand ups.]]></title>
            <link>https://medium.com/@siddhesh.jog/6-ways-to-get-the-most-out-of-your-stand-ups-40f78921d238?source=rss-95a776206365------2</link>
            <guid isPermaLink="false">https://medium.com/p/40f78921d238</guid>
            <category><![CDATA[agile]]></category>
            <category><![CDATA[stand-up]]></category>
            <category><![CDATA[scrum]]></category>
            <dc:creator><![CDATA[siddhesh jog]]></dc:creator>
            <pubDate>Tue, 23 Oct 2018 07:21:00 GMT</pubDate>
            <atom:updated>2018-10-23T11:41:30.793Z</atom:updated>
            <content:encoded><![CDATA[<h3>6 ways to get the most out of your stand-ups.</h3><p>My first experience of scrum came back in 2006. I was excited about working in a cool new way. Sprints, Backlog, sprint planning, stand-ups, demos were the words making the rounds. Yet it took little time for all of it to turn into a monotonous process. Quite the antithesis of what it was envisioned to be. Stand-ups, in particular, tend to become status update sessions rather than collaboration forums. You can visibly see the problem when developers are facing the manager when they speak. Even when there is some semblance of collaboration, the templated approach “what I did yesterday, What I will do today?” kills it. There is a lack of vision and slowly the whole thing becomes a chore. The stand up no longer serves its purpose and instead becomes a micromanagement platform. The fact that most scrum masters are Project Managers does not help as they can’t help but ask for status.</p><p><em>The Problems:</em></p><ul><li>Micromanaged status updates</li><li>a boring template approach</li><li>Lack of direction.</li></ul><p>It is not uncommon to see a stand up with some disinterested participants, some looking to score points and still, others going on about “What they did?” without considering the “How is this helpful?” aspect of their update.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/316/0*QnSozADjfz5UEIPS.png" /></figure><p>While there is no single remedy that can fix this 100% percent a few measures can help improve the situation significantly.</p><p><strong>1</strong>. <strong>Mr. Master Know your role: </strong>As a scrum master if you are actively participating in the stand-up; day in day out then know this: “You are a very large part of the problem”. The stand-up is a meeting for the team, of the team, by the team. A place and time to assess whether the sprint is on target and if required enlist the scrum master’s help to eradicate obstacles. If the scrum master is doing their job well, then they are essentially not required to speak in the stand-up. It is critical that a scrum master own the role and shed their status changing Project Manager skin.</p><p><strong>2. A Stand-up mission</strong>: it is vital that the sprint goal is established when planning and serve as the mission for the stand-up. Most training guides tell us that stand up is about discussing your current state of affairs through (“What did I do”, “What Will I do”, “What are my obstacles”). What needs to be established and ingrained is; <strong><em>why</em></strong> are we doing this? The stand-up template most prescribed focuses on the “what” and “how” instead of, the <strong><em>why</em></strong>.</p><p><strong>3. “<em>Your</em> stand up <em>your</em> template”</strong>: Let the team members decide how they want to conduct their stand up. As long as the stand up serves its purpose of “allowing the team to inspect, adapt and course correct daily”. Allowing the team to decide how they do stand up, will empower the team members, make them feel engaged and allow them to create a stand up that is efficient and enjoyable. Be it totems, tv screens projecting JIRA boards, or chairs turned around at desks; they all work if the team believes in the process &amp; follows it religiously.<br> <strong><br></strong> <strong>4. Punctuality, tools &amp; precision: </strong>A stand up must begin at the same time every day; must be attended by all healthy team members. The respect for the stand up is essentially respect for your colleagues and their time. Making use of the JIRA scrum boards and burndown charts can keep stand up time down. If the board is updated “everyone already knows what you did yesterday?”. The aim should be to keep the information shared with your teammates precise and succinct. Verbose updates can lead to wandering minds and kill interest.</p><p><strong>5. Horses for courses:</strong> To start with, get the right people. Look for self-motivated, assertive team members who are hungry to learn. At least a couple of team members who are ideators when it comes to processes and who are passionate about improvising and tailoring processes to meet the team’s needs. Their positive energy and contribution can rub off on others in the team and create just the right environment for the process to organically move in the right direction.</p><p><strong>6. A right mix of experience</strong>: A combination of senior and junior resources is a must. Generation after generation people grow up with ever-changing technology (in terms of tech, a generation for me is about 5 years) and the younger techies are generally more in touch with the ever-changing tech landscape. Moreover, I am a firm believer that radical innovation flows bottom-up from the junior most to the senior most. On the other hand, senior team members can bring in years of experience in taking ideas from inception to fruition. They know the common pitfalls and can guide the team through change. While it’s not easy to fix the two points above if you inherit a team (as a lead/team member/scrum master); you can definitely make course correction overtime to get the right balance.</p><p>Above all, it is important for the team members to own the stand-up; to believe in its use and ability to make a difference in their productivity and have fun <em>“standing up”</em>. The stand up is a key cog in the <em>“inspect and adapt”</em> machine and getting it right is imperative. These steps have helped me improve stand-ups (albeit not with total success). Here’s hoping they can serve you well too.</p><p><em>Originally published at </em><a href="http://sjtn.blogspot.com/2018/10/recipe-for-tasteful-standup.html"><em>sjtn.blogspot.com</em></a><em> on October 23, 2018.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=40f78921d238" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>