<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Andrei Contan on Medium]]></title>
        <description><![CDATA[Stories by Andrei Contan on Medium]]></description>
        <link>https://medium.com/@andreicontan?source=rss-15e2d38ed761------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*fS0xrHj_Y4ekGGyn.jpeg</url>
            <title>Stories by Andrei Contan on Medium</title>
            <link>https://medium.com/@andreicontan?source=rss-15e2d38ed761------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 18:56:43 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@andreicontan/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Achieving resilience in…software]]></title>
            <link>https://medium.com/@andreicontan/achieving-resilience-in-software-bd71513e201b?source=rss-15e2d38ed761------2</link>
            <guid isPermaLink="false">https://medium.com/p/bd71513e201b</guid>
            <category><![CDATA[performance-testing]]></category>
            <category><![CDATA[resilience]]></category>
            <category><![CDATA[software-testing]]></category>
            <category><![CDATA[resilience-engineering]]></category>
            <category><![CDATA[testing]]></category>
            <dc:creator><![CDATA[Andrei Contan]]></dc:creator>
            <pubDate>Thu, 21 Mar 2019 07:39:44 GMT</pubDate>
            <atom:updated>2019-03-21T10:30:28.930Z</atom:updated>
            <content:encoded><![CDATA[<h3>Achieving resiliency in…software</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xUPIvfThtieB4QIo3214eg.jpeg" /><figcaption>Illustration by Gizem Vural (source: <a href="https://www.newyorker.com/science/maria-konnikova/the-secret-formula-for-resilience">https://www.newyorker.com/science/maria-konnikova/the-secret-formula-for-resilience</a>)</figcaption></figure><p>What is resiliency and how do we achieve it? Can we see it, implement it, test it or validate it? What’s the price? What’s the effort? Resiliency means many things to many people, takes various shapes and definitions. In a scientific research on Resiliency, Manikam Pillay (<a href="https://doi.org/10.4236/ojsst.2017.74012">https://doi.org/10.4236/ojsst.2017.74012</a>) lists more than 15 definitions as a result of a systematic review. Often associated with Netflix and their efforts to contribute to the domain of Resilience Engineering, the topic is far from being covered.</p><p>Being involved in a similar effort of helping a company to achieve resiliency in its software, as a tester I’m facing the following challenge:</p><p>Having given:</p><blockquote><strong>Postulate 1</strong> — Jerry Weinberg — No matter how it looks at first, it’s always a people problem.</blockquote><p>and</p><blockquote><strong>Postulate 2 </strong>— Conway’s Law — organizations which design systems … are constrained to produce designs which are copies of the communication <a href="https://en.wikipedia.org/wiki/Organizational_structure">structures</a> of these organizations</blockquote><p>One could say that having a resilient software implies having a resilient company with resilient employees. So in order to achieve high resiliency, we need to focus on the people first and code/infra second.</p><p>Expanding on the resiliency topic to humans, I found out 3 basic principles:</p><p><strong>Principle 1</strong></p><blockquote>Resilience is first and foremost a property of environments that are A) rich in personal, social, material, and energy resources, B) allow access to those resources, and C) provide safety and protection against resource loss and promote resource growth. (<a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4826077/#R16">Hobfoll, 2012</a>, <em>p. 229</em>).</blockquote><p>So companies are likely to be resilient if A) hire valuable /skilled people, high social interaction and resources to achieve means B) provide easy access to those resources and C) builds a threat safe workplace, where employees are allowed to fail and explore new challenges</p><p><strong>Principle 2</strong></p><blockquote>The resources required for resiliency are acquired and aggregate across the lifespan (<a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4826077/#R16">Hobfoll, 2012</a>).</blockquote><p>In other words, over time the more resilient you are, more resilient you become. while being less resilient we’re most likely to become even less resilient.</p><p><strong>Principle 3</strong></p><blockquote>The accumulation of resource losses is more rapid and powerful than the accumulation of resource gains over time. Which means that it’s most likely to lose resiliency than to gain resiliency ((<a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4826077/#R20">Hobfoll &amp; Lilly, 199</a>3)</blockquote><p>Let’s stop for a second. So software resiliency is defined by how resilient the company, the workplace and their engineers are?</p><blockquote>That’s the “qualisophic” question</blockquote><p>Don’t have fact supported conclusion, but rather a theory and some observations. All I observe is that we, as humans, tend to do everything humanly possible to move the focus away from us. So instead of humans first, code second we find it more comfortable to do code first, humans second. Convenient, isn’t it? Let’s park the human aspect and address the testing part</p><p>Aside from the human element, how do we approach the resiliency problem?</p><p>Well, again Netflix seems to have an important insight which is presented here: <a href="https://www.infoq.com/presentations/failure-test-research-netflix">https://www.infoq.com/presentations/failure-test-research-netflix</a></p><p>Some of the most interesting ideas include:</p><ul><li>Redundancy is defaulted in the system overall (infrastructure — deployment into multiple availability zone, multiple regions)</li><li>Fault-tolerance includes redundancy</li><li>Implicit redundancy vs explicit redundancy =&gt; you need to aim for explicit redundancy</li><li>Analyse a success story and walk backwards and see what possible could have gone wrong.</li><li>Redundancy = GetBookmark(APICall) + GetBookmarkFallback (which usually returns an empty something in case the initial call fails to execute)</li><li>Idempotence does not hold in microservice architecture — who might know what has changed along the way</li><li>Request — response model in which you observe not only the services and components touched during processing, but also the order in which they fail with respect to the end result.</li><li>Not all services and nodes are interesting to fail</li><li>“Asking “if this component is down or not available, how will it continue to deliver value to our end user?” is a great place to start the design process. Defining the top five revenue-generating workflows for your business, then designing how you can ensure they are resilient, is a good strategy.”</li></ul><p>So how does a testing strategy looks like? Again, depends on the problem and what you define by resiliency. During the last months I managed to gather a list of questions and topics to address in conversation with peers. List includes copy paste from various sources, no reference unfortunately, please reference the source if you find it:</p><p><strong>SERVER SIZING</strong></p><ul><li>How many application servers are needed to support the customer base?</li><li>What is the optimal ratio of users to web servers?</li><li>What is the optimal web server–to–application server ratio?</li><li>What is the maximum number of users per server?</li><li>What is the maximum number of transactions per server?</li></ul><p><strong>SERVER TUNING AND OPTIMIZATION</strong></p><ul><li>Which specific hardware configurations provide the best performance?</li><li>How can vendor default configurations be tuned to suit this specific infrastructure and application?</li><li>What system resources need tuning to give optimal performance?</li></ul><p><strong>CAPACITY PLANNING</strong></p><ul><li>What is the current production server capability?</li><li>Is there room for growth?</li><li>What hardware or software can be added to achieve the next level of performance or capacity?</li><li>Is there excess capacity? Can a server be removed without compromising performance?</li></ul><p><strong>THIRD-PARTY VALIDATION</strong></p><ul><li>What is the current ISP and network capacity?</li><li>Can the ISP deliver on the service-level agreement that was signed?</li></ul><p><strong>SECURITY EXPOSURE</strong></p><ul><li>Can system vulnerabilities be identified and minimised?</li><li>What is the failover for firewalls?</li><li>Are there new vulnerabilities when excess user load is added to the application?</li><li>How susceptible is the system to DoS attacks?</li><li>If a DoS attack occurs, how will the system respond?</li><li>If the system goes down due to a hacker attack, how effective are the recovery procedures?</li></ul><p><strong>Infrastructure</strong></p><p>The following are some questions you should ask regarding the infrastructure.</p><p><strong>BROWSER/USER PROFILE ISSUES</strong></p><p>This subsystem is known as the <em>user community profile</em> and consists of business process definitions.</p><ul><li>What do the users do? (These are business-process definitions.)</li><li>How fast do the users do it? What are the transaction rates of each business process?</li><li>When do they do it? What time of day are most users using it?</li><li>What major geographic locations are they doing it from?</li><li>Is the application browser- or interface-dependent?</li><li>Is modem, WAN, or LAN emulation necessary?</li><li>Are there asynchronous communications between the browser/client and the backend servers?</li><li>Are there any non-HTTP(s) communications between the browser/client and backend servers?</li></ul><p><strong>INTERNET ISSUES</strong></p><ul><li>What are the peering issues associated with the client’s hosting/bandwidth provider?</li><li>What is the hosting strategy?</li></ul><p><strong>SITE WEB PIPE ISSUES</strong></p><ul><li>How much bandwidth does the site have?</li><li>Who is the client’s bandwidth provider? (Peering issues)</li><li>Are there multiple web pipes?</li></ul><p><strong>BORDER ROUTER ISSUES</strong></p><ul><li>What kind of load-balancing are the multiple pipes configured for?</li><li>Does it use the same inbound pipe as outbound pipe?</li><li>Is there equal distribution for outbound regardless of inbound pipe?</li><li>Is there the same outbound pipe regardless of the inbound pipe?</li><li>Are there multiple border routers?</li><li>What is the failover configuration for multiple border routers?</li></ul><p><strong>LOAD-BALANCE ISSUES</strong></p><ul><li>What type of load-balancing scheme is used? (Round robin, sticky IP, least connections, subnet based?)</li><li>What is the timeout of LB table?</li><li>Does it do any connection pooling?</li><li>Is it doing any content filtering?</li><li>Is it checking for HTTP response status?</li><li>Are there application dependencies associated with the LB timeout settings?</li><li>What failover strategies are employed?</li><li>What is the connection persistence timeout?</li><li>Are there application dependencies associated with the LB timeout settings?</li><li>What are the timeouts for critical functions?</li></ul><p><strong>PERIPHERAL SYSTEMS ISSUES</strong></p><ul><li>Is the LAN/WAN system dedicated or shared with other applications?</li><li>Are there any shared production resources?</li><li>Are there any web pipes, ERP systems, mail servers, filesystems, DNS servers, and so on?</li><li>Does it share databases with other applications?</li><li>Does it share hardware with other applications?</li></ul><p><strong>EXTERNAL SYSTEMS ISSUES</strong></p><ul><li>Are there any outside vendors that provide content distribution systems (CDS) for the architecture?</li></ul><p><strong>DISTRIBUTED HOSTING ISSUES</strong></p><ul><li>Are these multiple mirrored sites?</li><li>Is any site configured for failover operation?</li><li>How is the traffic load-balanced across the sites?</li><li>Are there architecture components on shared WAN connections?</li><li>What is the failover and recovery behavior?</li></ul><p><strong>FIREWALL ISSUES</strong></p><ul><li>What is the throughput capacity?</li><li>What is the connection capacity and rate?</li><li>What is the DMZ operation?</li><li>What are the throughput policies from a single IP?</li><li>What are the connection policies from a single IP?</li></ul><p><strong>IDS: INTRUSION DETECTION SYSTEMS</strong></p><ul><li>Is there statistical content sampling?</li><li>Is there an inverse relationship between throughput and security?</li><li>How is content filtering achieved?</li></ul><p><strong>Application</strong></p><p>Here are some of the questions you should be prepared to ask regarding the application.</p><p><strong>WEB SERVER ISSUES</strong></p><ul><li>How many connections can the server handle?</li><li>How many open file descriptors or handles is the server configured to handle?</li><li>How many processes or threads is the server configured to handle?</li><li>Does it release and renew threads and connections correctly?</li><li>How large is the server’s listen queue?</li><li>What is the server’s “page push” capacity?</li><li>What type of caching is done?</li><li>Is there any page construction done here?</li><li>Is there dynamic browsing?</li><li>What type of server-side scripting is done? (ASP, JSP, Perl, JavaScript, PHP, and so on)</li><li>Are there any SSL acceleration devices in front of the web server?</li><li>Are there any content caching devices in front of the web server?</li><li>Can server extensions and their functions be validated? (ASP, JSP, PHP, Perl, CGI, servlets, ISAPI filter/app, and so on)</li><li>Monitoring (Pools: threads, processes, connections, and so on; queues: ASP, sessions, and so on; general: CPU, memory, I/O, context switch rate, paging, and so on)</li></ul><p><strong>APPLICATION SERVER ISSUES</strong></p><ul><li>Is there any page construction done here?</li><li>How is session management done and what is the capacity?</li><li>Are there any clustered configurations?</li><li>Is there any load-balancing done?</li><li>If there is software load-balancing, which one is the load-balancer?</li><li>What is the page construction capacity?</li><li>Do components have a specific interface to peripheral and external systems?</li></ul><p><strong>DATABASE SERVER ISSUES</strong></p><ul><li>Have both small and large data sets been tested?</li><li>What is the connection pooling configuration?</li><li>What are its upper limits?</li></ul><p><strong>Security</strong></p><p>Here are some of the questions to ask when addressing security issues.</p><p><strong>FIREWALLS AND MULTIPLE DMZS</strong></p><ul><li>Does the firewall do content filtering?</li><li>Is it sensitive to inbound and/or outbound traffic?</li><li>What is its upper connection limit?</li><li>Are there policies associated with maximum connection or throughput per IP address?</li><li>Are there multiple firewalls in the architecture (multiple DMZs)?</li><li>If it has multiple DMZs, is it sensitive to data content?</li></ul><p><strong>IDS: INTRUSION DETECTION SYSTEM</strong></p><ul><li>Is there any content filtering?</li><li>Is the system sensitive to inbound and/or outbound traffic?</li><li>What are the alert thresholds?</li><li>What are the acceptable security thresholds</li></ul><p>Wow, you’ve made it this far and scroll throughout the entire list. For this, dear reader, I thank you. Let me know your thoughts on the resiliency topic, in any of the 2 aspects of the story — what do you think about the human resiliency theory?</p><p>What about the testing aspects? Feel free to add some more heuristics to the list.</p><p>Have fun!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bd71513e201b" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>