<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Nick Taranov | Fractional CTO on Medium]]></title>
        <description><![CDATA[Stories by Nick Taranov | Fractional CTO on Medium]]></description>
        <link>https://medium.com/@redpill-solutions?source=rss-93445ab8e6cf------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/2*zC-dbY26FC8kv1Y_h8Lz3Q.png</url>
            <title>Stories by Nick Taranov | Fractional CTO on Medium</title>
            <link>https://medium.com/@redpill-solutions?source=rss-93445ab8e6cf------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 14:29:46 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@redpill-solutions/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[The Types of CTOs Out There]]></title>
            <link>https://blog.venturemagazine.net/the-types-of-ctos-out-there-e6c64ce3119c?source=rss-93445ab8e6cf------2</link>
            <guid isPermaLink="false">https://medium.com/p/e6c64ce3119c</guid>
            <category><![CDATA[fractional-cto]]></category>
            <category><![CDATA[cto-as-a-service]]></category>
            <category><![CDATA[cto]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[startup]]></category>
            <dc:creator><![CDATA[Nick Taranov | Fractional CTO]]></dc:creator>
            <pubDate>Mon, 11 Mar 2024 04:55:26 GMT</pubDate>
            <atom:updated>2024-03-18T10:00:35.876Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*4bA9V77TiTWTuAmY" /><figcaption>Types of CTOs</figcaption></figure><p>I want to ask you about something right away. Please, don’t take the article too seriously. I mean, it’s not incorrect or anything, I just don’t want to be liable for any medical complications if you find something below eerily familiar.</p><h3>Windows Admin</h3><p>Remember the high schooler who helped with routers in a family-held food chain? He got promoted to the Chief Technology Officer of the entire enterprise. Or that smart student managing infrastructure for an entire strip mall college? He has “CTO” in his CV too! These are usually good and industrious types, but they won’t bother any startup founders. They represent a fair share of LinkedIn CTO population though. If, in addition to basic Excel, they go as far as learning PowerPoint, they transition into the next form.</p><h3>PowerPoint Hero</h3><p>The most common archetype is the CTO of a company where prime business is not software development. Maybe this is a bank equipment wholesale trader company, or land developer, or even an IT services company! They generally administrate stuff, but on a higher level or cater to their likes. The “IT Operations” CTO. You can tell them from the others by the level of PowerPoint they demonstrate. They perfected their decks for entire career!</p><p>Can they harm your startup prospects? Yes, they can, and they likely will. How? Being overly focused on operations, they tend to</p><ol><li>Hire too many people. I mean, they had an entire department at the previous place.</li><li>Procure too much stuff. Licenses, cloud infrastructure etc. All the 3-years offerings.</li><li>Implement every single best practice and ISO-69420 maturity standard.</li></ol><p>Mind you, I saw such types hiring multiple junior developers, and basically paying them a stipend while carrying oversized cloud costs, spending hundreds of thousands dollars on this while adding zero incremental value to the product.</p><p>Ironically, if somewhat realistic, they might outsource the entire development process to a software shop. What is the success rate with an external contractor is a case like this? I’d say 30% with a budget close to the initial, and 60% with a 3x budget. “Success” will be relative, as the code will be largely unmaintainable. If the product is extra simple, this might even work! See, not negative!</p><p>If you ask this type the “how come” question, you will be presented with a carefully crafted nuanced argument about A-B-C having been implemented. It has been the main game all along.</p><h3>Strategic Astronaut</h3><p>Sometimes, listening to a Silicon Valley CEO, you wonder if they are micro-dosing or not. You might not be sure about the CEO himself in this regard, but you can be pretty sure about their compensation — it’s high.</p><p>It appears that some CTOs and even Fractional CTOs (!) mimic this behavior. Cargo cults? Desire to look expensive? Micro-dosing? Maybe the dealer knows; I don’t.</p><p>Are you are going to benefit from a visionary CTO who loves to think about the technology 5, 10, 20 years ahead in 30,000 feet view? He might give some really influential interviews for your global multi billion innovative company, though.</p><p>This brings us to the next question. Are you running a Facebook or Twitter-size operation? Because CTOs of such caliber, who are somehow 100x more abundant than relevant positions, will not bother about all these nitty gritty details. They spend all their time (and your money) to think BIG. They “solve” things with ideas. Do you have AWS? AWS didn’t work? Try Azure? Azure didn’t solve the task? Try GCP. Rinse and repeat.</p><p>Microservices, DevOps, product management extremes, multiple experiments a day, multiple deployments a day, DDD, TDD, BDD, etc.</p><p>Awesome one of their kind highly specialized absolutely necessary consultants, who is so specialized, they cannot even explain what they do to commoners apart from being friends to the CTO.</p><p>End user value is not necessarily a part of the list. Daily grind? Control? Working with the backlog? Knowing what the subordinates are doing? God forbid, hands on? A VP of Engineering should do all this. What? You don’t have another $300,000 a year for a VP of Engineering? Hm-m….</p><p>How to recognize one?</p><ol><li>Close your eyes.</li><li>Soak up every word.</li><li>Does it sound like a leadership coach who’s read Wired for a week?</li></ol><p>If the answer is YES this is one.</p><p>This might be an option for a bigger company with a lot of ambition and money to waste, but your small startup will not benefit neither from their level of abstraction nor from their evangelism.</p><p>But, sure, they and their activities will not be cheap. But hey! You always wanted to sacrifice your business to help your CTO appear on a magazine cover, didn’t you?</p><h3>Founding Engineer</h3><p>Let’s start with a few simple questions.</p><ul><li>Are you building something extremely innovative? <br><strong>A tip</strong>: not a website, not a mobile app, not a platform or marketplace. Not Chinese vehicle recharge batteries with wheels showing ads, I saw 2 startups like this today. Maybe a principally new compression algorithm, like in the “The Silicon Valley” series. Think, pizza delivery drones at the very least.</li><li>Maybe, this is your partner, and he already has a patent or a prototype.</li><li>You or your startup are so exciting that investors line up to give you money with exorbitant valuations.</li></ul><p>If ANY of answers is YES, well, maybe a “classical” team with a Founding Engineer, and other engineers, and maybe a separate head of product is fine. Many successful startups were built like this.</p><p>If all 3 answers are NO, or NOT SURE, then maybe you can do better.</p><p>Now let’s pivot and think about something else. Okay, so you want a Founding Engineer. This is a demanded professional with high compensation expectations.</p><p>Some reality checks:</p><ul><li>How many Engineers are good with people? I’d say 30%. Here we enter Principal Developer salary territory.</li><li>How many of the 30% are good with management, planning, and organizing things? 50%? We are already in Software Manager compensation land. I mean, possibly upper 6 figures.</li><li>And, finally, how many of the above want to do unexciting stuff, and have all the stress? <br>“Running a start-up is like chewing glass and staring into the abyss. After a while, you stop staring, but the glass chewing never ends”, as Elon Musk formulated it.</li></ul><p>Okay, there is luck, and we all love to think that we are special, and rare good things happen to us, but bad things don’t. But even if you motivated such a rare bird to join your startup, what are the odds of such a talented and demanded person not leaving your company in a year? Or a month?</p><h3>What you actually want for an early stage startup</h3><p>You likely want to get from <strong>Point A </strong>to <strong>Point B</strong>.</p><p><strong>Point A </strong>— no MVP at all or a product which needs to be significantly improved to demo raising a round or serve the users. There is no development process in place or there are problems with the development.</p><p><strong>Point B — </strong>you have the MVP or the <strong>N+1</strong> version of your product good enough to raise a round or appealing enough to create traction. You are in control of the ongoing an further development.</p><p>How do you do that? Let’s step aside. <br>Do you know how they build houses nowadays? Like regular suburban houses, skyscrapers are a different story. There are special people in a builder company, who do pre-construction. Architectural projects, approvals, finding specialized contractors. A series of different and more specialized contractors then do “their own thing”. One does demolition, another does foundation, framing and drywall are oftentimes done by different contractors. Yet another one for electricity, one more for plumbing, who knows how many for finishes. Why do they build houses this way? Because they are smart and this is the most efficient way of doing this. What if I tell you that actually building your MVP like this is going to be better for small startups without The Great Wall or Eiffel Tower ambitions?</p><p>But why don’t you go one step further and just hire a software shop to finish your MVP? Well, there is saying “<a href="https://medium.com/p/e7177d567af3">No building since the Great Pyramid of Giza has been built on schedule and on budget</a>”. In fact, dedicated software shops might be capable of this. They have special people who understand risks and create right contracts (I personally created a bunch), and also special people able to control the contractors and keep them responsible. Hiccups do happen when working with new contractors, anyway.</p><p>You are not a software shop — your business model and product are different. And even if your product is somewhat innovative, chances are that 80% of it is not. Ideally you need top talent, but just 20–50% of the time, and to delegate all the not so challenging stuff to somebody else. So, for a non sci-fi tech business, you need a part-time CTO delegating 80% of tasks to cheaper resources. If you can afford to offer equity to make the person wake up at night thinking about your (and their) business, this works even better.</p><p>You want a person who can be a hands-on key player in the development and generally be your one-stop-everything tech center in your company.</p><p>Your prospective CTO should be able to do tasks like</p><ul><li>building the initial product — managing the tech side of it, panning, features, sprints, bugtracker etc;</li><li>coding, especially the parts that cannot be delegated;</li><li>hands-on problem-solving;</li><li>making efficient technical choices to set the development on right track and avoid bleeding money.</li></ul><p>So, that’s it. If you find the person, you are good to go until your operations grow significantly, and only then you might need something or someone else. This is about the time you can afford a yacht.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e6c64ce3119c" width="1" height="1" alt=""><hr><p><a href="https://blog.venturemagazine.net/the-types-of-ctos-out-there-e6c64ce3119c">The Types of CTOs Out There</a> was originally published in <a href="https://blog.venturemagazine.net/">Venture</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Cloud Costs Optimization Guide]]></title>
            <link>https://redpill-solutions.medium.com/cloud-costs-optimization-guide-7ecd4763f0c?source=rss-93445ab8e6cf------2</link>
            <guid isPermaLink="false">https://medium.com/p/7ecd4763f0c</guid>
            <category><![CDATA[cloud]]></category>
            <category><![CDATA[cost-optimization]]></category>
            <category><![CDATA[azure]]></category>
            <category><![CDATA[aws]]></category>
            <category><![CDATA[google-cloud-platform]]></category>
            <dc:creator><![CDATA[Nick Taranov | Fractional CTO]]></dc:creator>
            <pubDate>Wed, 31 Aug 2022 17:59:37 GMT</pubDate>
            <atom:updated>2022-08-31T18:20:15.250Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="Cloud Cost Optimization (Azure, GCP, AWS)" src="https://cdn-images-1.medium.com/max/613/1*VHAc1sAz4IrJj5sUNCVULA.png" /></figure><p>This article is a good starting point if you want to optimize your cloud costs, be it Microsoft Azure, Google Cloud Platform, Amazon Web Services, or another cloud provider. Unlike the situation with most similar articles, I’m not employed in a cloud or software vendor, so I don’t have the conflict of interests and am not going to promote any specific cloud service or tool. I do provide my own <a href="https://www.redpill.solutions/cloud-cost-optimization">consulting services</a> you might consider using, but the article can also be used as an end-to-end guide for your own efforts in cloud cost optimization.</p><h3>💡 Understand where costs come from</h3><h4>What do you pay for?</h4><p>In the cloud you pay for the right to use resources provided by someone else. These resources can take different forms, but generally we are speaking about the following types.</p><ul><li>Compute;</li><li>Storage;</li><li>Networking;</li><li>Software and added-value services either provided by the cloud vendor or third parties.</li></ul><h4>How do you pay for it?</h4><p>There are different pricing models, but commonly you pay for one of these.</p><ul><li>Reserved capacity;</li><li>Actual consumption;</li><li>Software licenses being included or not;</li><li>Features available or different pricing tier (Basic, Standard, Premium etc);</li><li>Conditions of use (SLA, Spot instances, Reserved instances etc).</li></ul><p>Also you might pay on different conditions. For example:</p><ul><li>Pay as you go;</li><li>Reserved/Prepaid/Savings plans;</li><li>Upfront monetary commitment plans.</li></ul><p>Obviously, consuming less vCPUs, RAM, storage or traffic might let you pay proportionally less or even less than that, but other details are also important. For example you might be able to switch your Windows Server virtual machine from “license included” pricing to “bring your own license” model and from pay-as-you-go to reserved instance and make your bill, say, 4 times smaller without changing the size of the VM.</p><p>Fine-tuning your cloud spending starts with gaining solid understanding of how exactly your costs are generated.</p><h3>🔍 Make the costs visible</h3><p>The goal here is to literally understand what costs and charges come from where. It’s easier if you start implementing your cloud with this visibility requirement in mind and might be more tedious if you let things drift by themselves for a significant amount of time or if you have a bigger cloud environment.</p><h4>Understand how the bill is structured</h4><p>Commonly, you receive an invoice from your cloud provider monthly. This bill has line items that might map to parts of your cloud environment or the terms of your contract with the provider. Please, also check if you have all your recurring costs mentioned in the monthly bill. Some services might have a longer billing period or there might be upfront costs you need to be aware of to judge how much you actually spend on the cloud.</p><p>Usually you have a higher-level abstraction that is used to aggregate costs for resources: subscriptions for <a href="https://docs.microsoft.com/en-us/azure/cost-management-billing/manage/mca-section-invoice">Azure</a> and projects for <a href="https://cloud.google.com/billing/docs/concepts">GCP</a>. For AWS the closest is <a href="https://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/billing-what-is.html">account</a>. Consider optimizing your invoice high level structure to make costs control easier, e.g., mapping Azure Subscriptions or GCP Projects to cost centers. With AWS you can use accounts to later consolidate the billing under Organization and use management accounts to oversee the costs.</p><p>You might also encounter resource-based charges or some other “service” charges like premium licenses for Azure Active Directory. You will need to research who is responsible for all these “services” costs and maintain your own records on this. A simple spreadsheet will do.</p><p>When it comes to “resources”, which will most often form the bulk of your costs, I explain some approaches below.</p><h4>Figure out who consumes what</h4><p>Having just a total amount, regardless of how big or small it is, doesn’t help in figuring out what money is actually spent on. Good news is that all the major cloud vendors provide ways to view costs based on some common characteristics. Here are the ones you can find pretty much everywhere.</p><ul><li>Location;</li><li>Resource type;</li><li>“Charge type” e.g. upfront fee vs monthly consumption based etc;</li><li>Tags/labels.</li></ul><p>“Dimensions” like location or resource type already might give you some general idea what you spend money on, but this might be not enough. Suppose, using an AWS example, all you know it’s EC2 and US East and this is the only region you use. To really get control over spending and understand your costs in terms which make sense to your business you need to leverage tags.</p><h4>Ode to Tags</h4><p>All major cloud providers also support a mechanic that actually lets you track what you spend on, in your terms. It’s called tags in Azure and AWS and labels in GCP, but it the nutshell they work the same for billing purposes. First you “tag” resources with your own custom tags, second you are able to see your expenditure split by tags. Good tag examples would be:</p><ul><li>Department;</li><li>Environment (e.g. Development, Staging, Production);</li><li>Cost center (in business sense);</li><li>Task-based resource groups (like all the resources supporting an application).</li></ul><p>Of course, you can come up with many more useful tag ideas to both track costs and help you govern your resources using automation or policies.</p><h4>Use the Tools</h4><p>Often tools provided by vendors are good and allow you to filter your costs based on the categories above, at least this is true for Azure, AWS, and GCP. Azure has “<a href="https://docs.microsoft.com/en-us/azure/cost-management-billing/cost-management-billing-overview">Cost Management + Billing</a>”, AWS has <a href="https://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/cost-alloc-tags.html">Billing Console</a> and GCP has <a href="https://cloud.google.com/billing/docs/how-to/reports">Cloud Billing Reports</a>.</p><p>Out-of-the-box tools are usually enough and additional tools are often not needed unless you know why you need them exactly.</p><h3>🔨 Act to minimize the costs</h3><p>We now know where the money is going, and what we want to minimize. Below, we will cover some actions that can be done to achieve it, but before we dig into the details I want to remind you about <a href="https://en.wikipedia.org/wiki/Pareto_principle">the 80/20 rule</a> which is very important for optimization tasks. The problem is that with a range of resources in your cloud environment, there is usually a wide range of actions that can be taken and strategies employed. The rule implies that you usually can get 80% of the results with 20% of effort. So the solution is to figure out what brings the best results for the time and money invested, and do just that.</p><h4>Deleting unused</h4><p>That’s it. You look around and find out what’s not used at all: unused virtual machines, orphan disks and backups, sometimes entire environments that are no longer needed or are not needed at the moment, and similar. You delete them — you save on them.</p><h4>Right-sizing</h4><p>Do you really need 4 virtual cores and 16 gigabytes of RAM for a small application? Maybe 800 Mb and a shared core will be enough? Inefficiencies like this commonly happen for two reasons. First, right-sizing of a resource is a tough task to do ahead of time when actual usage patterns are not exactly clear. Seriously, if you never reviewed and resized your resources, do it right now, it will save you a lot of money. Second, environments and configurations drift over time, and sometimes you find yourself with capacity over provisioned for a need that is no longer there. A noteworthy partial case of right sizing is creating or fine-tuning scale down rules for the resources that support auto scaling.</p><h4>Right-tiering</h4><p>Do you really need the “Premium” tier or “Basic” pricing tier will work? Consuming a tier too expensive for the job is generally harder to spot because it is often not about the amount of resources you consume but about the features you do not necessarily use or can do without. I would recommend going through your detailed bill items and ask yourself if you need the extra features or not.</p><h4>Changing resource locations</h4><p>Data centers are different. Because of this, cloud providers might bill different money for exactly the same resource depending on the location. Sometimes this is also a trade-off. Suppose, you can place your virtual machines in a cheaper location, but you will pay more for bandwidth. To come up with the right decisions, you need to do some “napkin math” and experiment with “price calculator” tool of your cloud provider (<a href="https://azure.microsoft.com/en-us/pricing/calculator/">Azure</a>, <a href="https://cloud.google.com/products/calculator">GCP</a>, <a href="https://calculator.aws/#/">AWS</a>) and also consider other things like latency.</p><h4>Scheduling and automaton</h4><p>Do you actually need to run that virtual machine 24/7? How much can you save turning it on and off? If the prospective savings justify it, you can use a script to do it automatically. You can also consider scaling resources vertically and horizontally on a schedule. Speaking about virtual machines specifically, cloud providers also support “spot instances” which can be evicted at any moment of time and are much cheaper. When used together with “restart automation” (there are many out-of-the-box and custom options), this might be an extremely cost-effective solution for a range of workloads like ETL, batch processing, training machine learning models, and similar long-running workloads.</p><h4>Reserved instances, savings plans and discounts</h4><p>All major cloud providers offer a better price if you commit for more. If you do need your resources for a prolonged period of time and can anticipate the utilization, this might enable significant savings, sometimes as large as 70% of the “pay as you go” price.</p><h4>Using another resource type or billing mode</h4><p>Sometimes you can rehost your application to another resource which is either cheaper or billed differently, e.g., on consumption basis instead of paying for capacity. Say, going from hosted Kubernetes (<a href="https://docs.microsoft.com/en-us/azure/aks/">AKS</a>, <a href="https://cloud.google.com/kubernetes-engine">GKE</a>, <a href="https://aws.amazon.com/eks/">EKS</a>) to a serverless offering supporting containers (<a href="https://azure.microsoft.com/en-us/services/container-apps/#overview">Azure Container Apps</a>, <a href="https://cloud.google.com/run">Google Cloud Run</a>, <a href="https://aws.amazon.com/fargate/?nc2=h_ql_prod_ct_far">AWS Fargate</a>).</p><h4>Grouping resources</h4><p>If your resources have different usage patterns or just don’t require much, they might be grouped together into a single Platform as a Service offering. Using database resources as an example, for Azure this would be <a href="https://docs.microsoft.com/en-us/azure/azure-sql/database/elastic-pool-overview?view=azuresql">SQL Elastic Pool</a>. This strategy can also be used with Infrastructure as a Service resources.</p><h4>Minimizing licenses consumed</h4><p>This is especially important for licenses which are included with cloud services. Sometimes, you might use your own license, sometimes you might avoid purchasing an extra one. Paying close attention to your licensing terms alone can easily result in saving thousands per month. Following up with the previous Azure Active Directory example, you might ask yourself if you need this exact number of, say Azure AD Premium P2 licenses or not. This might save a few dozen dollars a month here and there. In a few years, you might end up with thousands saved.</p><h4>Optimizing for requirements</h4><p>YAGNI — You Ain’t Gonna Need It. This is about re-thinking non-functional requirements for your applications and setting priorities. The cookie-cutter approach sure saves time, but do you really need the same resources you use for production in your development environment? Do you need a QA environment at all when you are not actively testing anything? Does this system actually need to be highly available or it was designed as such as a result of blindly following “best practices”? What is the target SLA for your application? Are resources and approaches used justified? Do you really need geographically distributed storage and premium network-level optimization? Understanding what you are actually doing is great to avoid spending too much.</p><h4>Taking advantage of free tiers</h4><p>All major providers want you to try different services, so they offer some limited use of selected resources for free. You can consider structuring your resources so that a significant portion fits below the free threshold. This savings strategy is probably the hardest to manage and has limited applicability, and this is why I list it the last.</p><h3>📉 Keep the costs down</h3><p>OK, so you reviewed your spending and took measures to minimize it, but how do you make sure it doesn’t go back up? Let’s put this straight. Controlling your costs is a management task and it cannot possibly be fully solved by a tool. You will need to have certain practices in place and dedicate your time to implement them. If you need training on how to do it, you can use the resources cloud providers give you (<a href="https://azure.microsoft.com/en-us/solutions/cost-optimization/#tools">Azure</a>, <a href="https://cloud.google.com/blog/products/it-ops/best-practices-for-optimizing-your-cloud-costs">GCP</a>, <a href="https://aws.amazon.com/aws-cost-management/aws-cost-optimization/">AWS</a>), or ask me for a custom training for your team. Here are things you can do to implement these practices yourself.</p><h4>Maintain costs visibility</h4><p>Keeping costs visible requires having certain things in place.</p><ul><li>Make costs tracking a part of your formal cloud strategy.</li><li>Assign the responsible for parts of your cloud environment. Use permissions to restrict who can spend money and where.</li><li>Create resources in well defined scopes.</li><li>Make sure resources are properly tagged/labeled, so that the costs could be told apart later.</li></ul><h4>Set up automated notifications and actions</h4><p>Some automation is already available, but you can also use your own custom scripts in more sophisticated scenarios.</p><ul><li>Use alerts to get notifications about expenditure above a certain threshold. Use multiple alerts triggered at different levels.</li><li>Set up budgets to be get alerted and shut down resources in an automated way when possible.</li><li>Limit autoscale unless you know exactly what you are doing. This is especially important when using serverless offerings as they are generally designed to scale in an explosive manner and can start to spend massive amounts of funds in a matter of minutes.</li></ul><h4>Review regularly</h4><p>Resources are provisioned by people and costs should be reviewed by people. This means that you might need to schedule regular reviews. You need to establish a FinOps process, agree about exact benchmarks and tools, and use them over and over again as a part of your business operations.</p><h3>💭 What’s next?</h3><p>So, you minimized the costs and have a plan to keep them down. How can you optimize your cloud costs even further? There are several ways.</p><h4>Consider migrating to/from on-premises or to another cloud</h4><p>Sometimes grass is actually greener on the other side. If your expected savings justify the <a href="https://www.redpill.solutions/consulting">consulting</a> project, you can actually move all or part of your workloads to where it’s cheaper to run them. It’s okay to be multi-cloud. It’s also okay to run something on-premises if it’s cheaper this way. All the major cloud providers support hybrid scenarios.</p><h4>Rearchitect to unlock new hosting options</h4><p>Let’s suppose you have a .NET Framework application and you run it in an Azure App Service or on a Windows virtual machine. If you rearchitect it to .NET Core (or 5+), you will be able to host it in a Linux container, using cheaper Kubernetes hosting option or a serverless offering like Azure Container Apps. Additionally, you will be able to host more applications using the same capacity, and rearchitecting a single application like this might easily result in savings of hundreds or even thousands of dollars per month.</p><h4>Optimize your applications</h4><p>Sometimes you can refactor your application to lower the usage of</p><ul><li>CPU;</li><li>Memory;</li><li>Storage;</li><li>Bandwidth;</li><li>Backend or database resources.</li></ul><p>Apart of just fixing inefficiencies, you do can:</p><ul><li>Compress data;</li><li>Use cache;</li><li>Eliminate same work being done in multiple places;</li><li>Use more effective protocols;</li><li>Change your containers’ base image to smaller and more efficient;</li><li>etc.</li></ul><h4>Automate managing capacity beyond out-of-the box</h4><p>Some resources are charged on usage basis but don’t have sufficient automated downscaling mechanics provided by the vendor. Think about things like shrinking capacity during off-hours for PaaS databases where possible and using spot instances with robust restart mechanics. <a href="https://aws.amazon.com/getting-started/hands-on/amazon-eks-with-spot-instances/">EKS</a>, <a href="https://docs.microsoft.com/en-us/azure/aks/spot-node-pool">AKS</a> and <a href="https://cloud.google.com/kubernetes-engine/docs/concepts/spot-vms">GKE</a> can be used with Spot VMs used as nodes automating their life cycle.</p><h4>Optimize for other important factors</h4><p>Your costs don’t necessarily come from the cloud provider bill. You can use more holistic view and try to minimize your operations costs in general.</p><ul><li>Optimize for lower maintenance/human labor consumption.</li><li>Minimize potential losses due to risks. Take into account the SLAs you have with your own clients and their financial implications.</li><li>Minimize the scope of your project.</li><li>Replace a part of your environment with SaaS.</li><li><a href="https://www.sdk.services/custom-training">Train your personnel FinOps</a> to keep your costs controlled.</li><li>Save money on spending too much time optimizing your environment! 😉</li></ul><h3>✅ Conclusion</h3><p>Now you know how to make your cloud costs visible, proactively minimize them, and keep them under control in the long run. There is no course of actions good for everyone. You might consider doing everything by yourself, or starting with <a href="https://www.sdk.services/custom-training">educating your employees</a>, or use <a href="https://www.redpill.solutions/consulting">professional services</a>. Regardless of what you decide to do, I wish you good luck in your savings journey. Do let me know if this guide proves to be useful for you.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7ecd4763f0c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Can You Hire a CTO for a Startup]]></title>
            <link>https://redpill-solutions.medium.com/can-you-hire-a-cto-for-a-startup-e7177d567af3?source=rss-93445ab8e6cf------2</link>
            <guid isPermaLink="false">https://medium.com/p/e7177d567af3</guid>
            <category><![CDATA[outsourcing]]></category>
            <category><![CDATA[development]]></category>
            <category><![CDATA[cto]]></category>
            <category><![CDATA[startup]]></category>
            <category><![CDATA[management]]></category>
            <dc:creator><![CDATA[Nick Taranov | Fractional CTO]]></dc:creator>
            <pubDate>Mon, 25 Jul 2022 16:13:59 GMT</pubDate>
            <atom:updated>2023-07-19T21:18:33.498Z</atom:updated>
            <content:encoded><![CDATA[<p>First of all, congratulations! 🎉 If you ended up here doing a web search like “CTO for startup” or “Technical Director for hire”, you are in a good place. I have been a software company CTO equivalent myself, so I will explain it to the point. So what Chief Technology Officer, or VP of Engineering, or whatever the role is called, does in a startup?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/512/0*BU3pdaGVuQlg4c-L.png" /></figure><p>CTO’s role, same as of all C-level executives, is conceptually really simple and so are the responsibilities. In software companies you at very least have two areas of responsibility — “business” and “technology”. <strong>The CTO is the person managing and being ultimately responsible for everything technology.</strong> This is it, you cannot create a comprehensive list of all the activities involved, this is by design.</p><p>Let’s suppose, you are the CEO.</p><p>👉 There is a need to evaluate Zooplazoopl technology released this morning combo with Flexible Scaled AgileOps — this is your CTO’s problem.</p><p>👉 You feel like you need to hire developers to move faster — discuss with the CTO, he’ll figure it out.</p><p>👉 The service is down — the CTO is responsible.</p><p>👈 The CTO tells you that the MVP is ready but nobody is interested — you are responsible.</p><p>👉 You need your company development gears to turn — good that the CTO takes care of it.</p><p>This gives you a fair idea of what CTO does.</p><h3>Is Hiring a Technical Director Just for Money Possible at All?</h3><p>TL;DR — no it’s not, but there is so much BS on the Internet about it.</p><p>Splitting roles in a startup is a business partnership, and just like marriage, it’s all about trust and doing each own part. Obviously, this cannot be decomposed into processes or elementary activities. It’s about doing whatever is necessary and knowing that the other person is doing their best in the same time. This is about motivation and being intrinsically interested in your product’s success.</p><p>There is another important part of the picture. Let’s suppose we have a “strong average” software shop of 20 decent developers. How many do you think are going to be CTO capable in terms of expertise, talent, ambition, and personal traits? My answer is something from 0 to 3. Being a software development director is nowhere near average, and just being a decent senior-level developer won’t make it. Add here that some people who actually can, do not want or no longer want to be responsible for the business function. One of my friends, and an ex-CTO too, left when faced with the need to fire employees not based on merit. People who are fit for the job are either busy being high-level technical managers in their own place with 2x-5x senior-level architect salaries or spending a couple of years in major tech companies waiting for their options to vest. They are working for boutique consultancies charging north of $500/h, fully analogous to what lawyers do. Some of them, those who feel entrepreneurial, are independent consultants like me. The rates are lower in this case as they don’t include the shop’s cut.</p><p>If you do a search for something like “Freelance CTO” you will find a plethora of articles on websites claiming to provide all sorts of services starting from “Fractional CTO” to “CTO for MVP” and such. Who are they? Just as observed above, they tend to be one of the following.</p><ol><li>Software shops doing projects on fixed scope or providing outsourced services.</li><li>Freelancer marketplaces.</li><li>Independent consultants.</li></ol><p>The articles on websites #1 and #2 are produced by marketing staff trying to scrap a long SEO text from sources on the Internet without having a clue at best or just key phrases stitched together at worst. They have no relation to reality and the best you can do is forget everything you read there. If you need an alternative definition, <a href="https://www.google.com/url?q=https%3A%2F%2Fen.wikipedia.org%2Fwiki%2FChief_technology_officer&amp;sa=D&amp;sntz=1&amp;usg=AOvVaw1ZAFMlaCFSJp09GNvf19lp">this is a better source</a>.</p><blockquote>“A <strong>chief technology officer</strong> (<strong>CTO</strong>), also known as a <strong>chief technical officer</strong> or <strong>chief technologist</strong>, is an<a href="https://www.google.com/url?q=https%3A%2F%2Fen.wikipedia.org%2Fwiki%2FSenior_management&amp;sa=D&amp;sntz=1&amp;usg=AOvVaw34v2LNDEHkNBuzAifJCKwk"> executive-level</a> position in a company or other entity whose occupation is focused on the scientific and technological issues within an organization.</blockquote><blockquote>A CTO is very similar to a<a href="https://www.google.com/url?q=https%3A%2F%2Fen.wikipedia.org%2Fwiki%2FChief_information_officer&amp;sa=D&amp;sntz=1&amp;usg=AOvVaw1Q2byLor4vfdLF-IekzH-W"> chief information officer</a> (CIO).</blockquote><blockquote>…</blockquote><blockquote>A CTO should be aware of new and existing technologies to guide the company’s future endeavors.”</blockquote><p>Speaking about #2 and #3 or Freelancers from marketplaces and independent consultants — there is nothing intrinsically wrong with them. In fact, I’m like this. You might find what you are looking for but success is not guaranteed. Is it the same as having a partner CTO? No.</p><h3>Is it Possible to Succeed Without a CTO?</h3><p>Yes, but <strong>if you don’t have a real CTO equity partner,</strong> <strong>actually you are the CTO</strong> even if you have a staff position with this name and a person within the team with this title. Equity partner motivation is different from employee’s and nothing can really fill the gap. But! You sure may still be able to do all parts yourself with the help of salary workers, outsources and consultants.</p><p>You are going to encounter some challenges though.</p><p><strong>Controlling the work of the outsourcing or in-house team</strong></p><p>Have you actually tried to manage a team of builders erecting your house? Controlling software devs is much more complex in both technical and business sense. You might structure your organization and processes just right, but this is another sort of challenge.</p><p>During many software development projects audits, I saw all sorts of shoddy and shady stuff, including mock HTML+JavaScript pages being presented as a finished functional subsystem with a 5 digits invoice being sent after the demo. I would say you need to be a rock star manager to generally make things work and a messiah to eliminate at least half of the slack if you don’t have extensive experience both in software development and managing the activity. Using external expertise and control can help as well.</p><p><strong>Setting right priorities and direction</strong></p><p>You might feel that you are naturally capable of doing it, huh? Suddenly, this might be even more challenging and dangerous. Let’s think about it together. For example, you are developing a web service application. You want to release a reasonably good project to secure your ability to work on it further. If this is your understanding, you might be in a good place already. A worse understanding would be to release something beating other products on the market. The terrible understanding would be trying to produce the best product in the world. Will a person interested in maximizing the scope give you this “just enough” advice? There are many aspects to it like technology used and who benefits from the choice, vendor or technology lock-in, or being too flexible and thus not productive, making your project bigger via inventing unnecessary problems and many more. There is a relevant joke, you know.</p><blockquote>The lawyer son runs into the home and yells to his lawyer dad: “Dad, do you remember the case you could not win our entire life the same as granddad before?”</blockquote><blockquote>“Yes, son” said the concerned lawyer.</blockquote><blockquote>“I just won it, and it was so easy that I can’t cannot believe how you weren’t able to do it for decades”</blockquote><blockquote>So the dad slapped his own face in despair and cried: “This case fed our family for generations, and you are coming here to my face to tell me you won it?!”</blockquote><p><strong>Maintaining the balance of power to remain in control</strong></p><p>“Hold on, you have to be joking. You are joking, right?”</p><p>No, I’m absolutely not. There is a proverb</p><blockquote>“No building since the Great Pyramid of Giza has been built on schedule and on budget”</blockquote><p>Often, this is not intended, but also often it is. Why? It’s as simple as one-two-three.</p><p>In construction:</p><ol><li>They sing sweet songs, and sign unrealistic plans to win versus the competition.</li><li>They demolish the previous building and dig a huge hole in the ground you can neither use nor leave.</li><li>Who’s the boss now?</li></ol><p>In software you have pretty much the same, in place of the hole in #2 it’s needing them to fulfill your own commitments in a timely manner, or only them knowing the changed code base now, or them being prepaid for work not yet done or other similar stuff. Anyway, you often cannot replace them without losing massively in time, or money, or you might no longer have the budget because they consumed it.</p><p>In the end you are not only in a dire situation, you also cannot really control them or make sure they do what it takes to release. Please be sure that those with dirty hands know exactly what they are doing. Suppose, they just billed you extra 500 hours for bug fixing. Did they even spend 50? This is especially common for the overseas teams.</p><p>How can you remediate this? Not going to lie, this is not something everybody will solve. Ideally you need experience both developing software and contracting dev services. You might use external expertise the same as you would in construction.</p><p><strong>Building what doesn’t work or is a bad idea</strong></p><p>Isn’t your idea good enough? It might be. More often it’s only true to some extent. Actually, this is where software is much more complex than construction and is more like engineering. Most startups pivot. Let’s suppose you are doing a lightweight MVP that was agreed to be 500 hours of work. It actually turns out to be more like 1500 hours. But! It’s also apparent for the dev team that things will not work as anticipated at all after 300 hours. These are realistic numbers. Will they tell you after 300 hours or wait till the 1500 hours of work is done?</p><p>There are, or course, ways to remediate this like having the experience yourself or using a (good!) consultant, using some agile methodology, and spending enough (much) time being involved with the product.</p><p>These are just probably the most important challenges, but there are others like being able to pick the right people, technologies, keeping it simple, being lean and such.</p><h3>Conclusion</h3><p>So, these were my 2 cents on some of the challenges you might encounter on your journey as a startup founder. I would recommend you to find a decent CTO and secure this healthy equity-based business co-owner motivation for them. <strong>If you decide to effectively be the CTO yourself, you will need to delegate. </strong>If what you just read resonates with you, you can contact me, the first consultation is free. If I won’t be able to help, I will just say so. If you solve the problem somehow in a way not covered here or do not agree, write me back, let’s have a meaningful discussion. No matter what you decide, I wish you persistence and luck in your endeavor! 🚀</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e7177d567af3" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Deploying to Kubernetes with GitLab]]></title>
            <link>https://redpill-solutions.medium.com/deploying-to-kubernetes-with-gitlab-28c2f1a42e57?source=rss-93445ab8e6cf------2</link>
            <guid isPermaLink="false">https://medium.com/p/28c2f1a42e57</guid>
            <category><![CDATA[git]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[continuous-delivery]]></category>
            <category><![CDATA[google-cloud-platform]]></category>
            <category><![CDATA[react]]></category>
            <dc:creator><![CDATA[Nick Taranov | Fractional CTO]]></dc:creator>
            <pubDate>Sun, 31 Oct 2021 14:38:26 GMT</pubDate>
            <atom:updated>2021-11-03T18:24:38.786Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/389/1*aQR52yluoZTOlddpnrN2cA.png" /></figure><p>This is the next article of a series started with the <a href="https://devops.redpill.solutions/articles/introduction-to-continuous-delivery-with-gitlab">previous tutorial on team work using GitLab</a>. The focus of the previous article was on continuous delivery in team work. This article provides practical guidance on deploying from GitLab to Kubernetes.</p><p>That is, we’ll take the simplest possible yet still contentual React.js application, dockerize it, then deploy it to Kubernetes locally using Docker Desktop. After, we’ll deploy it to Google Cloud Platform (GCP) and will follow up with designing a CI/CD pipeline in GitLab to publish our application to Google Kubernetes Engine (GKE).</p><p>You will benefit from having a basic understanding of</p><ul><li>Docker;</li><li>Kubernetes;</li><li>Git;</li><li>Node.js;</li><li>React;</li><li>Bash.</li></ul><p>We’ll do the following.</p><ul><li>🧱 Get familiar with our application, discuss its parts.</li><li>🐳 Dockerize our application.</li><li>☸️ Deploy our application to Kubernetes locally on Docker Desktop.</li><li>☁️ Discuss GCP features, and how we need to change the application. Then we once again deploy our application to Kubernetes, but this time to GCP.</li><li>🦊 Wrap up our tutorial by creating a pipeline for deploying the application to GCP using GitLab.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*JmTDZdVfc5C8gAV_s7bdMw.gif" /></figure><p>From time to time, I will ask you to do something. Such moments are marked with a 🛠️ sign. Please follow the steps as you read the text to get the most out of the tutorial.</p><p>⚙️ In order to go through this tutorial, you need these programs to be installed on your computer.</p><ul><li>Bash Shell</li><li>Git</li><li>Node.js</li><li>Docker Desktop</li><li>A modern web browser</li></ul><p>The commands in the text are in bash format, and some of them use the git command line client. If you are using Windows, the easiest way to get both is to install <a href="https://git-scm.com/downloads">Git for Windows</a>.</p><p>Further, we will assume that the directories with node, git, docker and kubectl binaries are included in PATH.</p><h3>🧱 Getting to know the app</h3><p>📢 Introduction to the application will include these steps.</p><ul><li>Clone the repository and find out what our application consists of.</li><li>Launch the application in the console.</li><li>See what the application will be in the future in the deployed state.</li></ul><h3>What the application consists of</h3><p>🛠️ Clone the repository and go to the application directory</p><pre>git clone <a href="https://github.com/ntaranov/gitlab-kubernetes">https://github.com/ntaranov/gitlab-kubernetes</a><br>cd gitlab-kubernetes</pre><p>The app has two parts</p><ul><li>Single Page Application (<strong>SPA</strong>) — a static React site that receives and modifies data by sending AJAX requests to the API service.</li><li><strong>API </strong>— a web service that provides data about user “registrations” and can add new data. The API writes data that needs to be persisted to a file. We use a file to avoid unnecessary complexity, which would be introduced by working with the database. Additionally, the file allows us highlight persistent volumes.</li></ul><p>Why to use such an application for the tutorial? Well, this is the simplest application that meets the following criteria.</p><ul><li>Uses a modern web framework(React.js) requiring code compilation.</li><li>Requires passing a parameter(API_URL) at the build time.</li><li>Contains server-side code.</li><li>Includes a Front End and API for emulating microservice architecture.</li><li>Has some persistent state.</li><li>Contains tests.</li></ul><p>I aimed to keep the application as simple as possible, so I deliberately didn’t include things like this:</p><ul><li>authentication and authorization;</li><li>logs and telemetry;</li><li>error handling;</li><li>interaction with database;</li><li>any meaningful test coverage.</li></ul><p>The goal of the tutorial is to provide a minimal example of deploying to Kubernetes and building a pipeline in GitLab. Even with all the simplifications, the tutorial ended up being quite huge.</p><p>Inside the newly cloned repository, there will be files and directories:</p><pre>api                 // Directory with API files<br>  /data/log         // We will save data here<br>  /package.json     // API NPM manifest<br>  /server.js        // API code, does not require installing other packages<br>spa                 // Directory with SPA files<br>  /public/.         // React standard static files<br>  /src<br>    /components<br>      /Log.jsx      // This component sends API requests<br>    /App.js         // Main SPA application component<br>    /App.test.js    // Tests, we must have some tests<br>    /config.js      // Loads an API URL from an environment variable<br>    /index.js       // React application root<br>  /package.json     // SPA NPM manifest<br>  /package-lock.json  // Worked with these versions of packages</pre><p>The rest of the files are either standard for create-react-app, or are secondary for our task.</p><p>Note that the component in the file ./spa/src/components/Log.jsx gets the value of the environment variable REACT_APP_API_URL using the code from ./spa/src/config.js. If nothing is passed, the code will use a default value good for running on local machine, but if we want to deploy the code elsewhere, we need to set the environment variable to a real value at the build stage, before executing npm run build.</p><h3>Launching locally</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*7FmskzJXgpuPenjA5lM5Aw.png" /></figure><p>Run <strong>API </strong>and <strong>SPA</strong>. The easiest way to do this is by opening two console windows.</p><p>🛠️ In the first window, run <strong>API</strong>.</p><pre>cd api<br>node server.js</pre><p>🛠️ In the second window, run <strong>SPA</strong>.</p><pre>cd spa</pre><p>Install <strong>npm </strong>packages according to package-lock.json.</p><pre>npm ci</pre><p><strong>SPA</strong> was created with create-react-app. Therefore, we will launch it using the development server from react-scripts.</p><pre>npm run start</pre><p>🛠️ Open <strong>SPA </strong>in your browser at</p><pre><a href="http://localhost:3000/">http://localhost:3000/</a></pre><p>🛠️ Click the <strong>Log Me!</strong> button. An AJAX request will be sent to the address http://localhost:3000/log and the User Agent data of the current user will be saved to the file ./data/log. The new data will be displayed on the page below the button.</p><p>Why does <strong>API </strong>running on port 4000 receive a request intended for http://localhost:3000/log? Actually, the create-react-app development server can work as a proxy, and redirects requests to http://localhost:4000/log. The corresponding setting is in ./spa/package.json, notice the last line&quot;proxy&quot;: &quot;http://localhost:4000&quot;.</p><p>You can run tests for</p><p>🛠️ Open SPA in your browser at</p><pre><a href="http://localhost:3000/">http://localhost:3000/</a></pre><p>🛠️ Click the Log Me! button. An AJAX request will be sent to the address http://localhost:3000/log and the User Agent data of the current user will be saved to the file ./data/log. The new data will be displayed on the page below the button.</p><p>Why does API running on port 4000 receive a request intended for http://localhost:3000/log? Actually, the create-react-app development server can work as a proxy, and redirects requests to http://localhost:4000/log. The corresponding setting is in ./spa/package.json, notice the last line&quot;proxy&quot;: &quot;http://localhost:4000&quot;.</p><p>You can run tests for <strong>SPA </strong>with the command</p><pre>npm run test</pre><p>You can prepare a static site for placement on production with the command</p><pre>npm run build</pre><h3>What do we want to see in the end?</h3><p>The target architecture can be displayed in this picture.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*6SUtDi6KgiBDTJC4.png" /></figure><p>For simplicity, we store data in a file, but if we move to storing data in a database, our architecture will become highly available.</p><p>First of all, we need to package our application as a Docker container image.</p><h3>🐳 Let’s dockerize our application</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*VapvinCPJWjbpswM.png" /></figure><p>A good introduction to Docker basics <a href="https://docs.docker.com/get-started/">available on the official site</a>.</p><p>📢 In order to dockerize our application, we first dockerize <strong>API </strong>and then <strong>SPA</strong>.</p><p>🛠️ Go to the directory ./api and create a Dockerfile with this code inside.</p><pre># use Alpine Linux to create a smaller image<br>FROM node: 16-alpine<br>WORKDIR /app<br>RUN mkdir data<br># copy the service code<br>COPY server.js ./<br>CMD [&quot;node&quot;, &quot;server.js&quot;]</pre><p>🛠️ Make sure Docker Desktop is running. Run local build and tag the image with gitlab-course-api</p><pre>docker build . -t gitlab-course-api</pre><p>🛠️ Run a container based on the image</p><pre>docker run --publish 4000:4000 -d gitlab-course-api</pre><p>This command will output the container hash, which you can use to stop the container later. Check that the service returns a test string when requesting <a href="http://localhost:4000/log.">http://localhost:4000/log.</a></p><h3>Dockerizing SPA</h3><p>🛠️ Same as with API let’s add Dockerfile to ./spa directory.</p><pre># build environment<br>FROM node:14-alpine as builder<br>WORKDIR /app<br>ENV PATH /app/node_modules/.bin:$PATH</pre><pre># copy React source codes into the build image<br>COPY package.json ./<br>COPY package-lock.json ./<br>COPY src ./src<br>COPY public ./public</pre><pre># install NPM packages according to package-lock.json<br>RUN npm ci</pre><pre># this param needs to be supplied as --build-arg when building the image<br>ARG API_URL<br># building the react application using the param<br>RUN REACT_APP_API_URL=$API_URL npm run build</pre><pre># production environment<br>FROM nginx:stable-alpine<br># copy the static site build result from our &quot;heavy&quot; build container<br># with NPM packages installed to a slim nginx web server image<br>COPY --from=builder /app/build /usr/share/nginx/html<br>EXPOSE 80<br>CMD [&quot;nginx&quot;, &quot;-g&quot;, &quot;daemon off;&quot;]</pre><p>In this case, Dockerfile ended up being more complex. In order to understand why there are such difficulties, let&#39;s understand the options we have when it comes to building <strong>SPA</strong>. We can build:</p><ul><li>locally;</li><li>on a dedicated build server;</li><li>directly in the container.</li></ul><p>In the case of a local <strong>SPA </strong>build, the build environment will differ from developer to developer. Nothing is wrong with the last two options, but we decided to choose the build in a container option to keep things simple. We want production image to be as small as possible and contain a minimum of software for stability and security reasons. Since our React application is just a static website, in production we only need a small image with a web server. However, to “build” our application, we need <strong>Node.js</strong> and all dependencies installed, which is a whole bunch of packages. To resolve this contradiction, we use a pattern called multi-stage build. We first build our application in a “heavy” container, which is essentially a build environment, and then copy only the resulting static site into a lightweight image, which will “serve” it in the production environment.</p><p>🛠️ Build the <strong>SPA </strong>image by running the following command in the newly created Dockerfile directory. Note that we are passing the <strong>API </strong>address as a parameter so that REACT_APP_API_URL variable is declared during the build process.</p><pre>docker build .  --build-arg API_URL=http://localhost:4000 -t  gitlab-course-spa</pre><p>Make sure the build completed successfully.</p><p>🛠️ Use the following command to start a container based on the newly created image.</p><pre>docker run -it --publish 80:80 gitlab-course-spa</pre><p>If port 80 is already taken on your computer, you can publish this container port on a different host port, for example, -p 8080:80.</p><p>🛠️ Open the <strong>SPA </strong>http://localhost page in your browser. You should see the same page as in your local test, however an attempt to call the <strong>API </strong>should fail with a <a href="https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS">CORS-related error</a>.</p><p>This is because from the point of view of the same origin policy/CORS the protocol, domain name and port must be the same, and if all of them do not match, then the sites are considered different. http://localhost:80 and http://localhost:4000 are different sites. Typically, in order to implement a scenario like ours, the <strong>SPA </strong>web server &quot;allows&quot; requests to the <strong>API </strong>by sending headers with the desired site to the user&#39;s browser.</p><p>💡 CORS is one of many topics, like setting up SSL/TLS, which we will not include in this tutorial to keep it from growing over 100 pages. At this stage, it was important for us to make sure that both web applications are working. When we deploy our application to <strong>Kubernetes</strong>, we will be able to receive requests to <strong>SPA </strong>and <strong>API </strong>on the same domain and port and route them based on the path.</p><p>💡 For local development, the docker-compose utility is often used. We could use it to implement the <strong>SPA </strong>and <strong>API </strong>interactions, as well as their interaction with the outside world through the auxiliary reverse proxy. docker-compose is closely related to the <a href="https://docs.docker.com/engine/swarm/">Docker Swarm</a> orchestrator , a <strong>Docker </strong>product, and uses configuration files with compatible syntax. Since we use <strong>Kubernetes </strong>as the orchestrator, we will run our application locally in <strong>Kubernetes </strong>without the extra step.</p><h3>☸️ Deploying to Kubernetes</h3><p>We’ve already learned how to run our application in Docker, now let’s move on to deploying to Kubernetes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*SSV1FY4l7HUNXK5d.png" /></figure><p>If you are not already familiar with Kubernetes, then I would recommend these materials.</p><ul><li><a href="https://kubernetes.io/docs/concepts/overview/what-is-kubernetes/">What is Kubernetes?</a></li><li>Other topics <a href="https://kubernetes.io/docs/concepts/overview/">Kubernetes Documentation -&gt; Concepts -&gt; Overview</a>.</li><li>Tutorial <a href="https://kubernetes.io/docs/tutorials/kubernetes-basics/">Learn Kubernetes Basics</a>.</li></ul><p>Kubernetes distribution, which comes with Docker Desktop, is usually sufficient to get started with the basics.</p><h3>Deploy the application locally to Kubernetes on Docker Desktop</h3><p>💡 If you cannot create all the necessary resources due to technical difficulties, you can either start debugging, which will require you delve into the details, or go to the next section where we will deploy our application to Kubernetes too but to Google Kubernetes Engine (GKE). In a different environment, the same technical problems may not arise.</p><p>Let’s create a directory where we will store Kubernetes resource files. Go to the root of the repository and run the command.</p><pre>mkdir kubernetes</pre><p>🛠️ Let’s start with <strong>SPA</strong>. Let’s add the file ./kubernetes/spa.yml with the following code.</p><pre>kind: Deployment<br>apiVersion: apps/v1<br>metadata:<br>  name: gitlab-course-spa<br>  namespace: gitlab-course<br>  labels:<br>    k8s-app: gitlab-course-spa<br>    project: gitlab-course<br>spec:<br>  replicas: 1<br>  selector:<br>    matchLabels:<br>      k8s-app: gitlab-course-spa<br>  template:<br>    metadata:<br>      name: gitlab-course-spa<br>      labels:<br>        k8s-app: gitlab-course-spa<br>    spec:<br>      containers:<br>      - name: gitlab-course-spa<br>        # this is how we named our docker image<br>        image: gitlab-course-spa<br>        imagePullPolicy: IfNotPresent<br>---<br>kind: Service<br>apiVersion: v1<br>metadata:<br>  name: gitlab-course-spa<br>  namespace: gitlab-course<br>  labels:<br>    k8s-app: gitlab-course-spa<br>    project: gitlab-course<br>spec:<br>  ports:<br>  - protocol: TCP<br>    port: 80<br>    targetPort: 80<br>  selector:<br>    k8s-app: gitlab-course-spa</pre><p>This file contains two YAML documents separated by ---.</p><ul><li>The first document defines <a href="https://kubernetes.io/docs/concepts/workloads/controllers/deployment/">Deployment</a> for <strong>SPA</strong>. Deployment in this case contains information about which Pods with which containers and in what number of replicas we want to run. Kubernetes relies on labels to understand which Pods belong to our Deployment. In this case, in the selector field, we indicate that these are Pods with k8s-app: gitlab-course-spa. We are building the image locally, so imagePullPolicy: IfNotPresent will use the local copy of the image when deploying locally.</li><li>The second document specifies <a href="https://kubernetes.io/docs/concepts/services-networking/service/">Service</a> for <strong>SPA</strong>. Service defines how our Pods will be available as a network resource.</li></ul><p>💡 We additionally define a project: gitlab-course label for all resources of our project so that we can later receive them all by a label request like this:</p><pre>kubectl get all --selector=project=gitlab-course -n gitlab-course</pre><p>🛠️ Looks like we have everything to deploy <strong>SPA</strong>, but there is a caveat. The fact is that we accessed the <strong>API </strong>at the address specified through the API_URL, and this value is embedded in the <strong>SPA </strong>image. Let&#39;s rebuild the image using the API_URL value, which we will use later.</p><pre>docker build ./spa --build-arg API_URL=http://spa.localtest.me/log -t gitlab-course-spa</pre><p>🛠️ Create <strong>SPA </strong>resources by running the command</p><pre>kubectl apply -f kubernetes/spa.yml</pre><p>Make sure the console confirms that both resources have been created.</p><p>💡 If you want to update the Pod image in Kubernetes, you will need to recreate the Pod. For <strong>SPA </strong>this can be done, for example, with this command.</p><pre>kubectl rollout restart deployment gitlab-course-api -n gitlab-course</pre><p>🛠️ Let’s move on to <strong>API</strong>. Let’s add the file ./kubernetes/api.yml with the following code.</p><pre>kind: Deployment<br>apiVersion: apps/v1<br>metadata:<br>  name: gitlab-course-api<br>  namespace: gitlab-course<br>  labels:<br>    k8s-app: gitlab-course-api<br>    project: gitlab-course<br>spec:<br>  replicas: 1<br>  selector:<br>    matchLabels:<br>      k8s-app: gitlab-course-api<br>  template:<br>    metadata:<br>      name: gitlab-course-api<br>      labels:<br>        k8s-app: gitlab-course-api<br>    spec:<br>      containers:<br>      - name: gitlab-course-api<br>        # this is how we named our docker image<br>        image: gitlab-course-api<br>        imagePullPolicy: IfNotPresent<br>        volumeMounts:<br>        - name: gitlab-course-pv<br>          mountPath: /app/data<br>      volumes:<br>      - name: gitlab-course-pv<br>        persistentVolumeClaim:<br>          claimName: gitlab-course-pv-claim<br>---<br>kind: Service<br>apiVersion: v1<br>metadata:<br>  name: gitlab-course-api<br>  namespace: gitlab-course<br>  labels:<br>    k8s-app: gitlab-course-api<br>    project: gitlab-course<br>spec:<br>  ports:<br>  - protocol: TCP<br>    port: 80<br>    targetPort: 4000<br>  selector:<br>    k8s-app: gitlab-course-api</pre><p>Everything is similar to <strong>SPA </strong>except for this part:</p><pre>...<br>        volumeMounts:<br>        - name: gitlab-course-pv<br>          mountPath: /app/data<br>      volumes:<br>      - name: gitlab-course-pv<br>          persistentVolumeClaim:<br>            claimName: gitlab-course-pv-claim<br>...</pre><p>The part is related to the fact that we use Persistent Volumes to allow the data stored by the <strong>API </strong>to live longer than the Pod.</p><ul><li>In volumes we declare a new volume and indicate that resources for it should be obtained using the Persistent Volume Claim gitlab-course-pv-claim.</li><li>In volumeMounts we mount this volume to /app/data directory where our application stores data.</li></ul><p>However, for this to work, we need the Persistent Volume Claim gitlab-course-pv-claim to have been defined ahead of the time.</p><p>🛠️ Create a new file ./kubernetes/volume.yml with this code.</p><pre>apiVersion: storage.k8s.io/v1<br>kind: StorageClass<br>metadata:<br>  name: local-storage<br>provisioner: kubernetes.io/no-provisioner<br>reclaimPolicy: Delete<br>volumeBindingMode: Immediate<br>---<br>apiVersion: v1<br>kind: PersistentVolume<br>metadata:<br>  name: gitlab-course-pv<br>  labels:<br>    k8s-app: gitlab-course-api<br>    project: gitlab-course<br>spec:<br>  capacity:<br>    storage: 100Mi<br>  volumeMode: Filesystem<br>  accessModes:<br>  - ReadWriteOnce<br>  persistentVolumeReclaimPolicy: Delete<br>  storageClassName: local-storage<br>  local:<br>    # the path is specific for Linux subsistem for Windows and maps to <br>    # C:\volumes\data\gitlab-course<br>    # this directory needs to be created before the creation of the volume <br>    path: &quot;/run/desktop/mnt/host/c/volumes/data/gitlab-course&quot;<br>  nodeAffinity:<br>    required:<br>      nodeSelectorTerms:<br>      - matchExpressions:<br>        - key: kubernetes.io/os<br>          operator: In<br>          values:<br>          - linux<br>---<br>kind: PersistentVolumeClaim<br>apiVersion: v1<br>metadata:<br>  name: gitlab-course-pv-claim<br>  labels:<br>    k8s-app: gitlab-course-api<br>    project: gitlab-course<br>spec:<br>  accessModes:<br>  - ReadWriteOnce<br>  resources:<br>    requests:<br>      storage: 100Mi<br>  storageClassName: local-storage<br>  volumeName: gitlab-course-pv</pre><p>This was perhaps the most boring moment of the tutorial; it&#39;s only getting better from now on. So, let&#39;s figure out what is here and why. Let&#39;s start with a PersistentVolumeClaim named gitlab-course-pv-claim. It defines a request for resources and we refer it in the <strong>SPA </strong>definition. Kubernetes is an extensible system, and the needs, in our case 100MiB, can be met in different ways, for example, by providing a cloud resource. Since we are publishing our application locally, we simply create a PersistentVolume named gitlab-course-pv just above. To create a PersistentVolume it is required, in turn, to specify a storage class that contains information about what kind of disk we need. Therefore, even above, we create a storage class with storageClassName: local-storage, which implies that the disk was created manually.</p><p>🛠️ Apply the above configuration.</p><pre>kubectl apply -f kubernetes/volume.yml</pre><p>Make sure the message in the console confirms that all three objects have been created.</p><p>🛠️ Now we have everything to create <strong>API </strong>objects.</p><pre>kubectl apply -f kubernetes/api.yml</pre><p>Our containers are already running in Kubernetes, but the services that we created are available only from the local software-defined network inside Kubernetes. One of the ways to provide access to our application from the outside is to create an object called Ingress. Ingress defines the rules by which our services will be available via HTTP.</p><p>💡 Make sure your version of Kubernetes has an ingress controller installed, for example community-supported ingress-nginx. If you cannot find the corresponding Deployment, you may need to <a href="https://kubernetes.github.io/ingress-nginx/deploy/">install it</a>. At the time of writing, the command for Docker Desktop looked like this</p><pre>kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.0.4/deploy/static/provider/cloud/deploy.yaml</pre><p>🛠️ Let&#39;s create a file kubernetes/ingress.yml with the following code.</p><pre>kind: Ingress<br>apiVersion: networking.k8s.io/v1<br>metadata:<br>  name: gitlab-course-ingress<br>  namespace: gitlab-course<br>  labels:<br>    project: gitlab-course<br>spec:<br>  rules:<br>    - host: spa.localtest.me<br>      http:<br>        paths:<br>          - path: /log<br>            pathType: Prefix<br>            backend:<br>              service:<br>                name: gitlab-course-api<br>                port:<br>                  number: 80<br>  defaultBackend:<br>    service:<br>      name: gitlab-course-spa<br>      port:<br>        number: 80</pre><p>In our case, as a result of creating the Ingress object, a reverse-proxy based on nginx will be brought up, which will transmit requests to the services of our application and responses from them back to the client. The gist of this configuration is that we accept requests on port 80 and redirect all requests to <strong>SPA </strong>except the requests to http://spa.localtest.me/log, which we route to <strong>API</strong>. We use localtest.me here - the domain name for which 127.0.0.1 IP address <a href="https://readme.localtest.me/">is returned for all the subdomains</a>.</p><p>🛠️ Create the Ingress resource.</p><pre>kubectl apply -f kubernetes/ingress.yml</pre><p>🛠️ Go to http://spa.localtest.me in your browser. It is important to use this specific domain because for another request to the <strong>API </strong>at the address http://spa.localtest.me/log will be blocked by CORS.</p><p>👍 Congratulations, you&#39;ve deployed your application to Kubernetes, albeit locally for now. As we will see later, using a different provider will not be much more difficult.</p><h3>☁️ Deploy our application to GCP</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*_hEsTlu61i_ndEa7.png" /></figure><h3>Creating a project in GitLab</h3><p>We will be using GitLab.com as our GitLab installation to avoid the hassle of installing and configuring the self-hosted version.</p><p>🛠️ If you don’t have a GitLab.com account yet, go to <a href="https://gitlab.com/">https://gitlab.com</a> and create one.</p><p>GitLab allows us to create a project by simply pushing to a remote repository.</p><p>🛠️ Let’s use the following command for this:</p><pre>GITLAB_USER_NAME=&lt;user name&gt;<br>git push https://gitlab.com/&lt;user name&gt;/gitlab-kubernetes</pre><p>&lt;user name&gt; here is your username on GitLab.com. Further we will consider the variable GITLAB_USER_NAME set to the same value in other scripts.</p><p>This command will create a private project named gitlab-kubernetes inside your GitLab.com account.</p><p>GitLab provides access to “deployment variables”, but to receive their values our branch must be “protected”. We will return to these variables later, but making sure that our master branch is protected is the most convenient for us right now.</p><p>🛠️ In the browser, go to your project page in GitLab https://gitlab.com/&lt;user name&gt;/gitlab-kubernetes;</p><ul><li>Select <strong>Settings </strong>-&gt; <strong>Repository </strong>from the menu on the left;</li><li>Expand the <strong>Protected branches</strong> submenu;</li><li>Make sure the master branch is listed and at least <strong>Maintainers </strong>are specified in the <strong>Allowed to push</strong> column (i.e. the value is not <strong>No one</strong>).</li><li>If the master branch is not listed, add it (<strong>Protect</strong>) using the above settings.</li></ul><h3>Creating a Kubernetes cluster in GKE via GitLab</h3><p>💡 To complete the tasks of this part, you may need to <a href="https://cloud.google.com/free">create a “trial” account</a>. You can get $300 USD to “play” with the resources during the trial period and <a href="https://about.gitlab.com/partners/technology-partners/google-cloud-platform/">additional $200 USD</a> if you use the GitLab’s offer. At least these were the proposals at time of writing.</p><p>💡 GitLab now also supports integration with Amazon EKS out of the box.</p><p>🛠️ Create a project called gitlab-kubernetes in Google Cloud Platform. If you haven&#39;t done this before, then <a href="https://cloud.google.com/resource-manager/docs/creating-managing-projects">see here how</a>.</p><p>💡 In fact, you can first create a cluster in Kubernetes, and then specify the key and token of the desired service account in GitLab, but this path is a bit longer and more likely to provoke errors, so we “cheat” and create a cluster in Kubernetes directly from GitLab, this way all the details will be filled in automatically.</p><p>🛠️ Create a Kubernetes cluster named gitlab-cluster-auto. The UI of GitLab.com is constantly changing, so the actions may be slightly different, follow the spirit, not the letter. At the moment, the actions are as follows:</p><ul><li>go to gitlab-kubernetes project;</li><li>in the left menu select <strong>Infrastructure </strong>-&gt; <strong>Kubernetes clusters</strong>;</li><li>click the <strong>Integrate with a cluster certificate</strong> button in the center of the page;</li><li>on the right, select the <strong>Create new cluster</strong> tab -&gt; <strong>Google GKE</strong>.</li></ul><p>Next, fill the form.</p><ul><li>Specify gitlab-cluster-auto as the cluster name.</li><li>Make sure the gitlab-kubernetes GCP project created for the course is selected.</li><li>Set <strong>Number of nodes</strong> to 1 — it is cheaper and <strong>API </strong>is not designed for more than one replica.</li><li>You can choose a smaller <strong>Machine type</strong>, for example n1-standard-1.</li><li>Leave the <strong>GitLab-managed cluster </strong>option checked.</li><li>Make sure <strong>Namespace per environment</strong> is set — we won’t be using many environments here, but it’s useful to enable this option because that’s closer to “real life”.</li></ul><p>Click the <strong>Create Kubernetes cluster</strong> button. Wait for the process to complete, make sure the <strong>Kubernetes cluster was successfully created</strong> message is displayed.</p><p>💡 At this stage, you may need to log into your GCP account and give GitLab the necessary permissions.</p><p>🛠️ Make sure there are no errors on the <strong>Kubernetes clusters</strong> page in gitlab-cluster-auto cluster list item.</p><p>👍 Now we have a Kubernetes cluster that we can use in the future.</p><h3>Deploying to Kubernetes on GCP “manually”</h3><p>📢 When working with a Kubernetes cluster in GCP, we will execute console commands in the <a href="https://shell.cloud.google.com/">Google Cloud Console Shell</a>. This will allow to</p><ul><li>simplify the authorization when accessing the cluster;</li><li>keep local kubectl settings the same.</li></ul><p>🛠️ Open the <a href="https://shell.cloud.google.com/">Google Cloud Shell</a> page in a browser.</p><p>🛠️ Google Cloud Shell has kubectl installed, but we need to&quot; connect &quot;it to the cluster. To do this, we need to specify the PROJECT_ID. Let&#39;s display a list of projects in our environment.</p><pre>gcloud projects list</pre><p>For example, for me PROJECT_ID was gitlab-kubernetes-326101. Assign this value to the PROJECT_ID variable</p><pre>PROJECT_ID=&lt;your value&gt;</pre><p>Now we can save cluster connection data for kubectl.</p><pre>gcloud container clusters get-credentials gitlab-cluster-auto --zone us-central1-a --project $PROJECT_ID</pre><p>Make sure the output was</p><pre>Fetching cluster endpoint and auth data.<br>kubeconfig entry generated for gitlab-cluster-auto.</pre><p>Create gitlab-course namespace</p><pre>kubectl create namespace gitlab-course</pre><p>Let’s set the variable GITLAB_USER_NAME in Google Cloud Shell too for convenience</p><pre>GITLAB_USER_NAME=&lt;user name&gt;</pre><p>📢 Let’s start deploying our application to GKE.</p><ol><li>Create a Persistent Volume Claim for the <strong>API</strong>.</li><li>Deploy <strong>API</strong>.</li></ol><ul><li>Upload <strong>API </strong>images to the GitLab repository.</li><li>Create <strong>API </strong>Deployment and Service.</li></ul><p>3. Deploy Ingress, it will be assigned an external IP address. The IP address is needed to specify <strong>API </strong>address when creating the <strong>SPA </strong>image.</p><ul><li>Testing <strong>API</strong>.</li></ul><p>4. Deploy <strong>SPA</strong>.</p><ul><li>Re-create the <strong>SPA </strong>image using the assigned IP address and upload the <strong>SPA</strong> image to the GitLab repository.</li><li>Finally, let’s create a Deployment and a Service for <strong>SPA</strong>.</li><li>Add a rule for <strong>SPA </strong>to Ingress, update Ingress. Test <strong>SPA </strong>and the entire application.</li></ul><h4>Creating Persistent Volume Claim for API</h4><p>💡 When working with cloud providers, you usually only need to create a Persistent Volume Claim, and the volume itself and the cloud resource that implements it are created automatically by the provider’s tools.</p><p>🛠️ Remove StorageClass and PersistentVolume in kubernetes/volume.yml. Remove storageClassName: local-storage line in the PersistentVolumeClaim definition, this will result in <a href="https://cloud.google.com/kubernetes-engine/docs/concepts/persistent-volumes">standard disk being used in GKE</a>. Also, replace the value of namespace with the token{{NAMESPACE}}.</p><pre>kind: PersistentVolumeClaim<br>apiVersion: v1<br>metadata:<br>  name: gitlab-course-pv-claim<br>  namespace: {{NAMESPACE}}<br>  labels:<br>    k8s-app: gitlab-course-api<br>    project: gitlab-course<br>spec:<br>  accessModes:<br>  - ReadWriteOnce<br>  resources:<br>    requests:<br>      storage: 100Mi</pre><p>💡 The idea is that namespace may differ depending on how and where we deploy the application. For this reason, we will replace {{TOKENS}} with actual values ​​before applying the configuration files. We could use something like Helm to work with templates, but we don&#39;t want to complicate the tutorial, do we?</p><p>🛠️ Upload this file to Google Cloud Shell. The <strong>Upload </strong>button is located on the command line menu in the Cloud Shell interface. At the time of this writing, the option was in the “<strong>⋮</strong>” submenu. As we tokenized the file, we need to replace the token with a real value first. For example, so</p><pre>sed &quot;s/{{NAMESPACE}}/gitlab-course/g&quot; volume.yml &gt; volume-replaced.yml</pre><p>Then apply the resulting configuration</p><pre>kubectl apply -f volume-replaced.yml</pre><p>Or you perform the substitution and apply the configuration in one line.</p><pre>sed &quot;s/{{NAMESPACE}}/gitlab-course/g&quot; volume.yml | kubectl apply -f -</pre><p>Check with the following command that the Persistent Volume Claim has been created.</p><pre>kubectl get persistentvolumeclaim --all-namespaces</pre><h4>API Deployment</h4><p>🛠️ Edit kubernetes/api.yml. Replace namespace and Deployment in Service this way.</p><pre>...<br>  namespace: {{NAMESPACE}}<br>...</pre><p>Let’s do the same with image, we will pass in the exact version of the Docker image to let Kubernetes know that the Pod needs to be updated.</p><pre>...<br>        image: {{API_IMAGE}}<br>...</pre><p>We can now substitute a different name for the image, but our local container image will not be accessed by GKE. We need to host our image in a service available to Kubernetes. Upload the local image to the GitLab Repository.</p><p>🛠️ From the <strong>local </strong>console, type</p><pre>docker login registry.gitlab.com/$GITLAB_USER_NAME/gitlab-kubernetes</pre><p>and enter you GitLab login data.</p><p>🛠️ Assign the tag with the prefix of our repository in GitLab to the image.</p><pre>docker tag gitlab-course-api registry.gitlab.com/$GITLAB_USER_NAME/gitlab-kubernetes/api</pre><p>🛠️ Now we can upload our image to the remote repository. Execute the command</p><pre>docker push registry.gitlab.com/$GITLAB_USER_NAME/gitlab-kubernetes/api</pre><p>Wait for the upload to finish and make sure it was successful.</p><p>🛠️ Type in <strong>Google Cloud Shell</strong></p><pre>docker login registry.gitlab.com/$GITLAB_USER_NAME/gitlab-kubernetes</pre><p>🛠️ There is a nuance that in order to download an image from our private repository on GitLab.com, you need to provide a secret. To do this, we can upload the secret that was created in ~/.docker/config.json when we ran the docker login command. Run this command in <strong>Google Cloud Shell</strong>.</p><pre>kubectl create secret generic regcred \<br>    --from-file=.dockerconfigjson=$HOME/.docker/config.json \<br>    --type=kubernetes.io/dockerconfigjson \<br>    --namespace=gitlab-course</pre><p>The last piece of the image configuration puzzle is to give our repository access to the secret information in kubernetes/api.yml. Place this code at the end of the Deployment definition, right after volumes:, on the same level as containers:</p><pre>...<br>      imagePullSecrets:<br>      - name: regcred<br>...</pre><p>On Google Cloud, we will use the default Ingress to route traffic from external IP to Pod in Kubernetes. Under the hood, Ingress implementation uses an External HTTP/S Load Balancer and requires all services to be available as NodePort. Let&#39;s change the service type in kubernetes / api.yml.</p><pre>...<br>spec:<br>  type: NodePort<br>...</pre><p>As a result, inside kubernetes/api.yml you should get the following code:</p><pre>kind: Deployment<br>apiVersion: apps/v1<br>metadata:<br>  name: gitlab-course-api<br>  namespace: {{NAMESPACE}}<br>  labels:<br>    k8s-app: gitlab-course-api<br>    project: gitlab-course<br>spec:<br>  replicas: 1<br>  selector:<br>    matchLabels:<br>      k8s-app: gitlab-course-api<br>  template:<br>    metadata:<br>      name: gitlab-course-api<br>      labels:<br>        k8s-app: gitlab-course-api<br>    spec:<br>      containers:<br>      - name: gitlab-course-api<br>        image: {{API_IMAGE}}<br>        imagePullPolicy: IfNotPresent<br>        volumeMounts:<br>        - name: gitlab-course-pv<br>          mountPath: /app/data<br>      volumes:<br>      - name: gitlab-course-pv<br>        persistentVolumeClaim:<br>          claimName: gitlab-course-pv-claim<br>      imagePullSecrets:<br>      - name: regcred<br>---<br>kind: Service<br>apiVersion: v1<br>metadata:<br>  name: gitlab-course-api<br>  namespace: {{NAMESPACE}}<br>  labels:<br>    k8s-app: gitlab-course-api<br>    project: gitlab-course<br>spec:<br>  type: NodePort<br>  ports:<br>  - protocol: TCP<br>    port: 80<br>    targetPort: 4000<br>  selector:<br>    k8s-app: gitlab-course-api</pre><p>Good! Now our image is available for Kubernetes and we can create a Pod!</p><p>🛠️ Upload the file kubernetes/api.yml to Google Cloud Console and run the command.</p><pre>sed -e &#39;s/{{NAMESPACE}}/gitlab-course/g&#39; -e &quot;s~{{API_IMAGE}}~registry.gitlab.com/$GITLAB_USER_NAME/gitlab-kubernetes/api~g&quot; api.yml | kubectl apply -f -</pre><p>Make sure the resources were created successfully.</p><p>💡 Note that we use ~ as a separator in the second sed expression because the image name already contains/.</p><p>💡 If you want to update the Pod image in Kubernetes, you will need to recreate the Pod. For <strong>API </strong>this can be done, for example, with this command.</p><pre>kubectl rollout restart deployment gitlab-course-api -n gitlab-course</pre><p>In our case, you will need to execute kubectl apply again, specifying sha256 explicitly adding the construction @sha256:&lt;hash&gt; to the image name.</p><h4>Deploying Ingress</h4><p>💡 We will use <a href="https://nip.io/">nip.io</a> to avoid having to register a domain. The problem here is that in order to specify the correct host name for the <strong>API </strong>in the Ingress definition, we need to know the external IP address of the Ingress. However, the IP address will be assigned to Ingress only after we create Ingress. The solution is to create the Ingress and then modify it.</p><p>🛠️ Let&#39;s create an Ingress. Modify the file kubernetes/ingress.yml.</p><pre>kind: Ingress<br>apiVersion: networking.k8s.io/v1<br>metadata:<br>  name: gitlab-course-ingress<br>  namespace: {{NAMESPACE}}<br>  labels:<br>    project: gitlab-course<br>spec:<br>  defaultBackend:<br>    service:<br>      name: gitlab-course-api<br>      port:<br>        number: 80</pre><p>💡 To create an Ingress, we need to specify either at least one rule or a default backend. We are temporarily specifying the <strong>API </strong>service because it has already been created.</p><p>🛠️ Upload this file to Google Cloud Shell.</p><pre>sed &quot;s/{{NAMESPACE}}/gitlab-course/g&quot; ingress.yml | kubectl apply -f -</pre><p>🛠️ Let&#39;s wait till the external IP is assigned to our Ingress. It is convenient to do this running this command</p><pre>kubectl get ingress --all-namespaces --watch</pre><p>Stop executing the command with Ctrl + C when the external IP is assigned. Assign it to the variable EXTERNAL_IP</p><pre>EXTERNAL_IP=&lt;external IP obtained by the previous command&gt;</pre><p>For simplicity, I will also call the address value itself EXTERNAL_IP.</p><p>🛠️ Go to http://EXTERNAL_IP.nip.io in your browser. For example, while testing this course, the address for me looked like http://34.102.175.33.nip.io/log</p><p>Make sure you receive a response with the code 200 OK.</p><p>So, we are almost finished deploying our application in GKE, it remains to deploy <strong>SPA </strong>and update Ingress.</p><h4>Deploying SPA</h4><p>🛠️ Remember the nuance with <strong>SPA</strong>? The <strong>API </strong>address is embedded inside the image, specified via the API_URL. The good news is that we have already created the Ingress and therefore we know the address that we can use as the API_URL. Let&#39;s rebuild the <strong>SPA </strong>image using this address. Run this command in your local console.</p><pre>docker build ./spa --build-arg API_URL=&quot;http://$EXTERNAL_IP.nip.io&quot; -t registry.gitlab.com/$GITLAB_USER_NAME/gitlab-kubernetes/spa<br>docker push registry.gitlab.com/$GITLAB_USER_NAME/gitlab-kubernetes/spa</pre><p>🛠️ Edit kubernetes/spa.yml. These will be the same edits as in kubernetes/api.yml before.</p><p>Replace namespace in the Deployment and in the Service this way.</p><pre>...<br>  namespace: {{NAMESPACE}}<br>...</pre><p>Let&#39;s do the same with image.</p><pre>...<br>        image: {{SPA_IMAGE}}<br>...</pre><p>Change the service type in kubernetes/spa.yml to NodePort.</p><pre>...<br>spec:<br>  type: NodePort<br>...</pre><p>Place the secret for accessing the GitLab repository at the end of the Deployment definition, on the same indentation level as containers:</p><pre>...<br>      imagePullSecrets:<br>      - name: regcred<br>...</pre><p>As a result, it should look like this:</p><pre>kind: Deployment<br>apiVersion: apps/v1<br>metadata:<br>  name: gitlab-course-spa<br>  namespace: {{NAMESPACE}}<br>  labels:<br>    k8s-app: gitlab-course-spa<br>    project: gitlab-course<br>spec:<br>  replicas: 1<br>  selector:<br>    matchLabels:<br>      k8s-app: gitlab-course-spa<br>  template:<br>    metadata:<br>      name: gitlab-course-spa<br>      labels:<br>        k8s-app: gitlab-course-spa<br>    spec:<br>      containers:<br>      - name: gitlab-course-spa<br>        image: {{SPA_IMAGE}}<br>        imagePullPolicy: IfNotPresent<br>      imagePullSecrets:<br>      - name: regcred<br>---<br>kind: Service<br>apiVersion: v1<br>metadata:<br>  name: gitlab-course-spa<br>  namespace: {{NAMESPACE}}<br>  labels:<br>    k8s-app: gitlab-course-spa<br>    project: gitlab-course<br>spec:<br>  type: NodePort<br>  ports:<br>  - protocol: TCP<br>    port: 80<br>    targetPort: 80<br>  selector:<br>    k8s-app: gitlab-course-spa</pre><p>🛠️ Download kubernetes/spa.yml into the ~/gitlab-course Cloud Shell directory and run the command.</p><pre>sed -e &#39;s/{{NAMESPACE}}/gitlab-course/g&#39; -e &quot;s~{{SPA_IMAGE}}~registry.gitlab.com/$GITLAB_USER_NAME/gitlab-kubernetes/spa~g&quot; spa.yml | kubectl apply -f -</pre><p>🛠️ It remains to update Ingress - and we&#39;re done. Let&#39;s do it now. Specify <strong>SPA</strong> as the default service and route requests to the <strong>API </strong>if the path is /log. Note that we are using Ingress by default in GKE, and therefore we are specifying pathType: ImplementationSpecific.</p><pre>kind: Ingress<br>apiVersion: networking.k8s.io/v1<br>metadata:<br>  name: gitlab-course-ingress<br>  namespace: {{NAMESPACE}}<br>  labels:<br>    project: gitlab-course<br>spec:<br>  rules:<br>    - host: {{EXTERNAL_IP}}.nip.io<br>      http:<br>        paths:<br>          - path: /log<br>            pathType: ImplementationSpecific<br>            backend:<br>              service:<br>                name: gitlab-course-api<br>                port:<br>                  number: 80<br>  defaultBackend:<br>    service:<br>      name: gitlab-course-spa<br>      port:<br>        number: 80</pre><p>🛠️ Delete ingress.yml file <strong>in the Google Cloud Shell</strong> and then re-upload kubernetes/ingress.yml file to the Google Cloud Shell and apply the changes with the command.</p><pre>sed -e &#39;s/{{NAMESPACE}}/gitlab-course/g&#39; -e &quot;s/{{EXTERNAL_IP}}/$EXTERNAL_IP/g&quot; ingress.yml | kubectl apply -f -</pre><p>🛠️ Go to the EXTERNAL_IP.nip.io page in your browser and make sure that our application works as expected. You may need to wait for a while as the Ingress configuration is getting updated in GCP.</p><p>👍 Congratulations, you have deployed your application to Kubernetes on the Google Cloud Platform! It&#39;s left only to implement the deployment of the application in GitLab.</p><h3>🦊 Deploying to Kubernetes with GitLab</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*L-98HArKUp-fNTY7.png" /></figure><p>The difficult part is over, but we still need to develop a pipeline in GitLab. So, let’s begin.</p><p>💡 We will not show here how to publish an application using AutoDevOps because AutoDevOps and everything related to it changes very often, and also because it is not really suitable for anything more complex than Hello World!. However, I recommend going through the <a href="https://docs.gitlab.com/ee/topics/autodevops/quick_start_guide.html">official tutorial</a> to get an impression of the built-in features of GitLab to use them when needed in the future.</p><p>💡 I don’t think it’s realistic to provide a one-size-fits-all solution, as the more “production” the pipeline is, the more complex it is, and the more it depends on the platform to which the deployment is done. For this reason, we will not strive to create a highly optimized pipeline, but instead focus on creating a solution that is easy to understand and that demonstrates different capabilities.</p><p>📢 The plan is</p><ul><li>Delete the resources created in the previous step</li><li>Set environment variables</li><li>Implement building and testing</li><li>Implement the deployment in GKE</li></ul><h3>Let’s delete the resources created in the previous step</h3><p>🛠️ We will delete resources “from top to bottom”. Run these commands in Google Cloud Shell.</p><pre>kubectl delete ingress gitlab-course-ingress -n gitlab-course<br>kubectl delete service gitlab-course-api -n gitlab-course<br>kubectl delete service gitlab-course-spa -n gitlab-course<br>kubectl delete deployment gitlab-course-api -n gitlab-course<br>kubectl delete deployment gitlab-course-spa -n gitlab-course<br>kubectl delete pvc gitlab-course-pv-claim -n gitlab-course<br>kubectl delete secret regcred -n gitlab-course</pre><h3>Setting environment variables</h3><p>🛠️ So, our pipeline will use EXTERNAL_IP variable, which will contain the external IP of our Ingress. Let&#39;s set its value inside GitLab.</p><ul><li>Open our GitLab project gitlab-kubernetes in the browser.</li><li>From the left menu in the <strong>Settings </strong>submenu, select <strong>CI</strong>/<strong>CD</strong>.</li><li>Expand the <strong>Variables </strong>submenu.</li><li>Press the <strong>Add Variable</strong> button. Enter EXTERNAL_IP in the <strong>Key </strong>field, and the value you received in the previous part in the <strong>Value </strong>field. In fact, we deleted the Ingress and will recreate it and there is no guarantee that GCP will assign the same IP to the Ingress we&#39;ll create soon, but it is highly likely, and if it does not, we will easily fix it.</li><li>Click the <strong>Add Variable</strong> button at the bottom of the form.</li></ul><p>Done!</p><p>If you are debugging your pipeline, you might also want to add CI_DEBUG_TRACE variable with the value of true. If you do this, the values ​​of all variables and parameters will be output in the job log.</p><p>💡 It’s a security risk to have CI_DEBUG_TRACE always on.</p><h3>Implementing building and testing</h3><p>🛠️ Create file .gitlab-ci.yml in the root of the project. Add this code</p><pre>image: docker:stable<br>services:<br># we will build our docker images inside a Docker container<br>- docker:dind<br>variables:<br>  DOCKER_DRIVER: overlay2<br>  SPA_DOCKER_IMAGE: $CI_REGISTRY_IMAGE/spa<br>  SPA_DOCKER_BUILDER_IMAGE: $CI_REGISTRY_IMAGE/spa-build<br>  API_DOCKER_IMAGE: $CI_REGISTRY_IMAGE/api</pre><p>DOCKER_DRIVER: overlay2 - According to <a href="https://docs.gitlab.com/ee/ci/docker/using_docker_build.html">documentation</a>, overlay2 is more efficient than vfs. This is actually the default for shared runners, but we left that line in case you want to use self-hosted runners.</p><pre>  SPA_DOCKER_IMAGE: $CI_REGISTRY_IMAGE/spa<br>  SPA_DOCKER_BUILDER_IMAGE: $CI_REGISTRY_IMAGE/spa-build<br>  API_DOCKER_IMAGE: $CI_REGISTRY_IMAGE/api</pre><p>We have introduced these variables for our convenience. These are the names of our images inside the GitLab registry. Why do we only have one variable for the <strong>API</strong>, but two for <strong>SPA</strong>?</p><p>For <strong>API </strong>we just wrap server.js in a small Node.js image.</p><p>For <strong>SPA</strong>, as you remember, we did a two-stage build.</p><ul><li>On the one hand, we want the image in production to be as small as possible.</li><li>On the other hand, in our CD pipeline we will run autotests, and we need NPM modules installed for this, i.e. for this we need a “larger” image.</li></ul><p>The solution is to split our two-stage build into a build of 2 images — “larger” and “smaller”.</p><p>🛠️ By the way, let’s do it right now.</p><p>Create file Build.Dockerfile and copy the first piece of code from Dockerfile into it</p><pre># build environment<br>FROM node:14-alpine as builder<br>WORKDIR /app<br>ENV PATH /app/node_modules/.bin:$PATH<br><br># copy React source codes into the build image<br>COPY package.json ./<br>COPY package-lock.json ./<br>COPY src ./src<br>COPY public ./public<br><br># install NPM packages according to package-lock.json<br>RUN npm ci<br><br># this param needs to be supplied as --build-arg when building the image<br>ARG API_URL<br># building the react application using the param<br>RUN REACT_APP_API_URL=$API_URL npm run build</pre><p>We will shorten the original Dockerfile. Let&#39;s delete the above code from the file and parameterize the name of the image from which we will eventually copy the files.</p><pre>ARG build<br># build environment<br>FROM $build as builder<br><br># production environment<br>FROM nginx:stable-alpine<br># copy the static site build result from our &quot;heavy&quot; build container<br># with NPM packages installed to a slim nginx web server image<br>COPY --from=builder /app/build /usr/share/nginx/html<br>EXPOSE 80<br>CMD [&quot;nginx&quot;, &quot;-g&quot;, &quot;daemon off;&quot;]</pre><p>As you see, simple and logical.</p><p>Add one more, third Test.Dockerfile file, in which we will run tests</p><pre># test environment<br>FROM spa-build:latest<br>RUN npm run test</pre><p>Yes, two lines only.</p><p>💡 Alternatively, we could re-use the “heavy” build image passing the run command via the command line.</p><p>Now everything is ready to implement all the stages associated with building and testing. 🛠️ Add this fairly large chunk of code to .gitlab-ci.yml.</p><pre>stages:<br>- build<br>- test<br>- package</pre><pre>before_script:<br>  - docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY</pre><pre>build_spa_builder:<br>  stage: build<br>  script:<br>  - docker build -f ./spa/Build.Dockerfile --build-arg API_URL=&quot;http://$EXTERNAL_IP.nip.io&quot; -t $SPA_DOCKER_BUILDER_IMAGE:$CI_COMMIT_SHORT_SHA ./spa<br>  - docker tag $SPA_DOCKER_BUILDER_IMAGE:$CI_COMMIT_SHORT_SHA  $SPA_DOCKER_BUILDER_IMAGE:latest<br>  - docker push $SPA_DOCKER_BUILDER_IMAGE:$CI_COMMIT_SHORT_SHA<br>  - docker push $SPA_DOCKER_BUILDER_IMAGE:latest</pre><pre>test_spa:<br>  stage: test<br>  script:<br>  - docker run $SPA_DOCKER_BUILDER_IMAGE:latest sh -c &quot;CI=true npm test&quot;<br>  dependencies:<br>  - build_spa_builder</pre><pre>build_spa:<br>  stage: package<br>  script:<br>  - docker build -f ./spa/Dockerfile --build-arg build=$SPA_DOCKER_BUILDER_IMAGE:latest -t $SPA_DOCKER_IMAGE:$CI_COMMIT_SHORT_SHA ./spa<br>  - docker tag  $SPA_DOCKER_IMAGE:$CI_COMMIT_SHORT_SHA $SPA_DOCKER_IMAGE:latest<br>  - docker push $SPA_DOCKER_IMAGE:$CI_COMMIT_SHORT_SHA<br>  - docker push $SPA_DOCKER_IMAGE:latest<br>  dependencies:<br>  - build_spa_builder</pre><pre>build_api:<br>  stage: package<br>  script:<br>  - docker build -f ./api/Dockerfile -t $API_DOCKER_IMAGE:$CI_COMMIT_SHORT_SHA ./api<br>  - docker tag $API_DOCKER_IMAGE:$CI_COMMIT_SHORT_SHA $API_DOCKER_IMAGE:latest<br>  - docker push $API_DOCKER_IMAGE:$CI_COMMIT_SHORT_SHA<br>  - docker push $API_DOCKER_IMAGE:latest</pre><p>This code defines three stages build, test and package. These stages, in turn, contain jobs. Let&#39;s discuss which job does what.</p><p><strong>build</strong></p><ul><li><strong>build_spa_builder </strong>— create the “build” image</li></ul><p><strong>test</strong></p><ul><li><strong>test_spa </strong>— test <strong>SPA </strong>code</li></ul><p><strong>package</strong></p><ul><li><strong>build_spa </strong>— copy <strong>SPA </strong>files to the slim production image</li><li><strong>build_api </strong>— copy <strong>API </strong>files</li></ul><p>The commands specified in before_script will be executed before each job. We use docker login to authorize the runner to access the GitLab Docker Registry using data available through special GitLab environment variables.</p><pre>before_script:<br>  - docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY</pre><p>We are using some variables here that start with CI_, these are the so-called <a href="https://docs.gitlab.com/ee/ci/variables/">GitLab CI/CD variables</a>. Their values are provided to us by the platform.</p><p>Most jobs above are structured similarly except for <strong>test_spa </strong>which is simpler because of not uploading the image to the repository. If you want to make sure that the previous job completes before starting the next job, you can specify the previous one in the dependencies. Let&#39;s analyze the commands using <strong>build_spa_builder </strong>as an example.</p><pre>script:<br>  - docker build -f ./spa/Build.Dockerfile --build-arg API_URL=&quot;http://$EXTERNAL_IP.nip.io&quot; -t $SPA_DOCKER_BUILDER_IMAGE:$CI_COMMIT_SHORT_SHA ./spa<br>  - docker tag $SPA_DOCKER_BUILDER_IMAGE:$CI_COMMIT_SHORT_SHA  $SPA_DOCKER_BUILDER_IMAGE:latest<br>  - docker push $SPA_DOCKER_BUILDER_IMAGE:$CI_COMMIT_SHORT_SHA<br>  - docker push $SPA_DOCKER_BUILDER_IMAGE:latest</pre><p>Here we create an image, then tag it with a label consisting of the value of our variable and the short hash of the current commit in the GitLab repository CI_COMMIT_SHORT_SHA. We then upload the image to the GitLab Docker repository. When creating the <strong>SPA </strong>image, we also use the EXTERNAL_IP variable that we set earlier.</p><p>There are two purposes of tagging the image with the hash of the commit.</p><ul><li>First, we will understand from which version of the code the image is built.</li><li>Secondly, for images that will be referenced inside YAML files, Kubernetes will be able to understand that the image has changed and that the Pod needs to be updated.</li></ul><p>🛠️ So, the moment of magic when we first run our pipeline has come. Update the code in the GitLab repository with the following command in the local console. Add all the code you haven’t added to the index yet, commit and push.</p><pre>git add -A<br>git commit -m &quot;Commit all the code remaining to build images with GitLab&quot;<br>git push</pre><p>🛠️ Open the <strong>CI/CD</strong> -&gt; <strong>Pipelines </strong>page in GitLab in a browser and make sure the build and testing has started. Go to our specific build’s page and wait for it to complete successfully. During this, it won’t hurt if you open the pages of different jobs and watch the updated output in the console.</p><p>Perfect! At this moment, we are already building all the images we need straight in GitLab.</p><h3>Implementing Deployment in GKE</h3><p>🛠️ Let’s take the last step — implement the deployment to Kubernetes on GCP using GitLab. So add the extra stage to .gitlab-ci.yml.</p><pre>stages:<br>- build<br>- test<br>- package<br>- deploy</pre><p>Then add the code after the build steps.</p><pre>deploy_spa:<br>  stage: deploy<br>  image: &quot;registry.gitlab.com/gitlab-org/cluster-integration/auto-deploy-image&quot;<br>  # we need to set the environment here, otherwise GitLab will not pass the values of variables starting with KUBE_<br>  environment: production<br>  before_script:<br>  - |<br>    kubectl create secret -n &quot;$KUBE_NAMESPACE&quot; \<br>    docker-registry regcred \<br>    --docker-server=&quot;$CI_REGISTRY&quot; \<br>    --docker-username=&quot;${CI_DEPLOY_USER:-$CI_REGISTRY_USER}&quot; \<br>    --docker-password=&quot;${CI_DEPLOY_PASSWORD:-$CI_REGISTRY_PASSWORD}&quot; \<br>    --docker-email=&quot;$GITLAB_USER_EMAIL&quot; \<br>    -o yaml --dry-run | kubectl replace -n &quot;$KUBE_NAMESPACE&quot; --force -f -<br>  script:<br>  # creating the volume<br>  - sed -e &quot;s/{{NAMESPACE}}/$KUBE_NAMESPACE/g&quot; kubernetes/volume.yml | kubectl apply -f -<br>  # deploying API<br>  - sed -e &quot;s/{{NAMESPACE}}/$KUBE_NAMESPACE/g&quot; -e &quot;s~{{API_IMAGE}}~$API_DOCKER_IMAGE:$CI_COMMIT_SHORT_SHA~g&quot; kubernetes/api.yml | kubectl apply -f -<br>  # deploying SPA<br>  - sed -e &quot;s/{{NAMESPACE}}/$KUBE_NAMESPACE/g&quot; -e &quot;s~{{SPA_IMAGE}}~$SPA_DOCKER_IMAGE:$CI_COMMIT_SHORT_SHA~g&quot; kubernetes/spa.yml | kubectl apply -f -<br>  # updating ingress<br>  - sed -e &quot;s/{{NAMESPACE}}/$KUBE_NAMESPACE/g&quot; -e &quot;s/{{EXTERNAL_IP}}/$EXTERNAL_IP/g&quot; kubernetes/ingress.yml | kubectl apply -f -</pre><p>Noteworthy here is the fact that instead of docker: stable we are using a different image. Generally, any image with kubectl would work for us, so for simplicity we use registry.gitlab.com/gitlab-org/cluster-integration/auto-deploy-image image which GitLab uses to deploy to Kubernetes in AutoDevOps mode.</p><p>We also <a href="https://docs.gitlab.com/ee/user/project/clusters/deploy_to_cluster.html#deployment-variables">take advantage of the fact that our cluster is integrated with GitLab</a>. For this reason, we can rely on environment variables that start with KUBE_ to be passed automatically.</p><p>We override the before_scripts.</p><ul><li>We do not want to try to log into docker before executing our job commands.</li><li>We want to give our Pods the ability to download images from the GitLab Docker Registry.</li></ul><p>Here is the complete code for .gitlab-ci.yml that should have resulted from the edits.</p><pre>image: docker:stable<br>services:<br>- docker:dind<br>variables:<br>  DOCKER_DRIVER: overlay2<br>  SPA_DOCKER_IMAGE: $CI_REGISTRY_IMAGE/spa<br>  SPA_DOCKER_BUILDER_IMAGE: $CI_REGISTRY_IMAGE/spa-build<br>  API_DOCKER_IMAGE: $CI_REGISTRY_IMAGE/api</pre><pre>stages:<br>- build<br>- test<br>- package<br>- deploy</pre><pre>before_script:<br>- docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY</pre><pre>build_spa_builder:<br>  stage: build<br>  script:<br>  - docker build -f ./spa/Build.Dockerfile --build-arg API_URL=&quot;http://$EXTERNAL_IP.nip.io&quot; -t $SPA_DOCKER_BUILDER_IMAGE:$CI_COMMIT_SHORT_SHA ./spa<br>  - docker tag $SPA_DOCKER_BUILDER_IMAGE:$CI_COMMIT_SHORT_SHA  $SPA_DOCKER_BUILDER_IMAGE:latest<br>  - docker push $SPA_DOCKER_BUILDER_IMAGE:$CI_COMMIT_SHORT_SHA<br>  - docker push $SPA_DOCKER_BUILDER_IMAGE:latest</pre><pre>test_spa:<br>  stage: test<br>  script:<br>  - docker run $SPA_DOCKER_BUILDER_IMAGE:latest sh -c &quot;CI=true npm test&quot;<br>  dependencies:<br>  - build_spa_builder</pre><pre>build_spa:<br>  stage: package<br>  script:<br>  - docker build -f ./spa/Dockerfile --build-arg build=$SPA_DOCKER_BUILDER_IMAGE:latest -t $SPA_DOCKER_IMAGE:$CI_COMMIT_SHORT_SHA ./spa<br>  - docker tag  $SPA_DOCKER_IMAGE:$CI_COMMIT_SHORT_SHA $SPA_DOCKER_IMAGE:latest<br>  - docker push $SPA_DOCKER_IMAGE:$CI_COMMIT_SHORT_SHA<br>  - docker push $SPA_DOCKER_IMAGE:latest<br>  dependencies:<br>  - build_spa_builder</pre><pre>build_api:<br>  stage: package<br>  script:<br>  - docker build -f ./api/Dockerfile -t $API_DOCKER_IMAGE:$CI_COMMIT_SHORT_SHA ./api<br>  - docker tag $API_DOCKER_IMAGE:$CI_COMMIT_SHORT_SHA $API_DOCKER_IMAGE:latest<br>  - docker push $API_DOCKER_IMAGE:$CI_COMMIT_SHORT_SHA<br>  - docker push $API_DOCKER_IMAGE:latest</pre><pre>deploy_spa:<br>  stage: deploy<br>  image: &quot;registry.gitlab.com/gitlab-org/cluster-integration/auto-deploy-image&quot;<br>  # we need to set the environment here, otherwise GitLab will not pass the values of variables starting with KUBE_<br>  environment: production<br>  before_script:<br>  - |<br>    kubectl create secret -n &quot;$KUBE_NAMESPACE&quot; \<br>    docker-registry regcred \<br>    --docker-server=&quot;$CI_REGISTRY&quot; \<br>    --docker-username=&quot;${CI_DEPLOY_USER:-$CI_REGISTRY_USER}&quot; \<br>    --docker-password=&quot;${CI_DEPLOY_PASSWORD:-$CI_REGISTRY_PASSWORD}&quot; \<br>    --docker-email=&quot;$GITLAB_USER_EMAIL&quot; \<br>    -o yaml --dry-run | kubectl replace -n &quot;$KUBE_NAMESPACE&quot; --force -f -<br>  script:<br>  # creating the volume<br>  - sed -e &quot;s/{{NAMESPACE}}/$KUBE_NAMESPACE/g&quot; kubernetes/volume.yml | kubectl apply -f -<br>  # deploying API<br>  - sed -e &quot;s/{{NAMESPACE}}/$KUBE_NAMESPACE/g&quot; -e &quot;s~{{API_IMAGE}}~$API_DOCKER_IMAGE:$CI_COMMIT_SHORT_SHA~g&quot; kubernetes/api.yml | kubectl apply -f -<br>  # deploying SPA<br>  - sed -e &quot;s/{{NAMESPACE}}/$KUBE_NAMESPACE/g&quot; -e &quot;s~{{SPA_IMAGE}}~$SPA_DOCKER_IMAGE:$CI_COMMIT_SHORT_SHA~g&quot; kubernetes/spa.yml | kubectl apply -f -<br>  # updating ingress<br>  - sed -e &quot;s/{{NAMESPACE}}/$KUBE_NAMESPACE/g&quot; -e &quot;s/{{EXTERNAL_IP}}/$EXTERNAL_IP/g&quot; kubernetes/ingress.yml | kubectl apply -f -</pre><p>🛠️ Commit the changes and push to the repository</p><pre>git add -A<br>git commit -m &quot;Commit the code to deploy to GKE with GitLab&quot;<br>git push</pre><p>Go to http://EXTERNAL_IP.nip.io in the browser and make sure the application works. You may need to wait a few minutes for all the resources to be updated.</p><p>💡 If the application does not open, but Deployments run without errors, especially if you get a 404 code when you go to the address above, it is likely that GCP assigned a different external IP to the newly created Ingress. It’s easy to fix!</p><ul><li>Find out the new IP address</li></ul><pre>kubectl get ingress -n &lt;namespace created by GitLab&gt;</pre><ul><li>Change the value of the EXTERNAL_IP variable on GitLab.com accordingly.</li><li>Restart the pipeline in GitLab.</li></ul><h3>🎉 Congratulations!</h3><p>You first dockerized an application in React.js, then manually deployed it to Kubernetes, and finished by creating a CD pipeline deploying the app to Kubernetes using GitLab!</p><p>🧹 Don’t forget to delete all the unnecessary resources created on Google Cloud Platform.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=28c2f1a42e57" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Introduction to Continuous Delivery with GitLab]]></title>
            <link>https://redpill-solutions.medium.com/introduction-to-continuous-delivery-with-gitlab-74872d5e2d96?source=rss-93445ab8e6cf------2</link>
            <guid isPermaLink="false">https://medium.com/p/74872d5e2d96</guid>
            <category><![CDATA[continuous-delivery]]></category>
            <category><![CDATA[javascript]]></category>
            <category><![CDATA[git]]></category>
            <category><![CDATA[gitlab]]></category>
            <category><![CDATA[react]]></category>
            <dc:creator><![CDATA[Nick Taranov | Fractional CTO]]></dc:creator>
            <pubDate>Wed, 31 Mar 2021 02:01:27 GMT</pubDate>
            <atom:updated>2021-03-31T02:01:27.488Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/400/1*I9Vy5DoZyPyWVPROQXCsiQ.jpeg" /></figure><p>The tutorial will give you a quick impression of how teamwork is done with GitLab. Generally, it’s easier to start practicing DevOps/CD with GitLab than with other products because GitLab is an all-in-one solution.</p><p>💻 During the tutorial, we will</p><ul><li>set up basic project management using GitLab;</li><li>create continuous delivery pipeline;</li><li>perform a couple of GitLab Flow cycles;</li><li>study CI/CD metrics in GitLab.</li></ul><p>📖 It’s good but not necessary to know</p><ul><li>Git;</li><li>Node.js;</li><li>React;</li><li>Docker.</li></ul><p>I will ask you to do something from time to time. Such actions are marked with 🛠️ emoji. Please perform such actions as you read the text to take the most from the tutorial.</p><h3>🏁 Intro and getting familiar with the project</h3><p>We will use a slightly modified <strong>create-react-app</strong> project as a guinea pig.</p><blockquote><em>Why React? First, it’s is the most widely adopted JavaScript UI library, and many readers are familiar with it. Second, </em><strong><em>create-react-app</em></strong><em> gives us sensible build and test stages which are already implemented.</em></blockquote><p>🛠️ Let’s now clone the repository with the code we are going to work with.</p><pre>git clone <a href="https://github.com/ntaranov/gitlab-cd-react">https://github.com/ntaranov/gitlab-cd-react</a></pre><p>🛠️ Navigate to the local repository folder</p><pre>cd gitlab-cd-react</pre><blockquote><em>You can just create new </em><strong><em>create-react-app</em></strong><em> application instead, but the version in the repo contains some fixes, and I tested the package versions.</em></blockquote><p>🛠️ Install the <strong>npm</strong> packages</p><pre>npm install</pre><blockquote><em>Use </em><em>npm ci instead if you encounter an error running </em><em>npm install</em></blockquote><p>🛠️ Try to &quot;build&quot; the project</p><pre>npm run build</pre><p>Note that ./build directory now has relevant files including the minified JavaScript and CSS.</p><p>🛠️ Then run the tests</p><pre>npm run test -- --coverage --watchAll=false --forceExit</pre><p>🛠️ Now we need to deploy to finish, but to avoid cluttering the tutorial with web server installation instructions, let&#39;s just launch the debug web server to see how our web application looks in browser.</p><pre>npm start</pre><p>Standard <strong>create-react-app</strong> starter page should appear.</p><p>If you think about it, all that remains is to replace the last instruction with copying the static files from ./build directory to a web server, and the deployment is done. Basically, this is enough to work by yourself, however, if you want to work as a part of a team, you might also need at least the following:</p><ol><li>centralized repository with</li></ol><ul><li>control over team members’ permissions;</li><li>code review;</li></ul><p>2. bug tracker to</p><ul><li>plan work and</li><li>track the progress;</li></ul><p>3. build system with</p><ul><li>pipelines launched automatically as needed;</li><li>build status visualization;</li><li>artifacts storage;</li><li>automatic notifications about key events;</li><li>automatic deployment into different environments;</li></ul><p>4. continuous deployment metrics collection.</p><p>All the above, and much more, is supported by Gitlab. <a href="https://about.gitlab.com/pricing/">The paid version</a> offers more features, but the free version available on <a href="https://gitlab.com/">GitLab.com</a> is enough for what we are after.</p><blockquote><em>What sets GitLab apart is the absence of restrictions on self-hosted job runners, however some pretty basic “management” and “corporate” GitLab features like mandatory approval for merge requests are considered premium offerings. Open source projects have access to all the GitLab features for free.</em></blockquote><h3>⚙️ Setting up basic project management on GitLab.com</h3><p>In this section we will create the GitLab project itself and issues we will work on later, and will set up a Kanban board to visualize the project status.</p><h4>💼 Creating a project in GitLab</h4><p>We will use GitLab.com to avoid the chores inevitable when installing and setting a self-hosted instance.</p><p>🛠️ Please browse to https://gitlab.com and register an account if you don&#39;t have one yet.</p><p>GitLab lets us create a project by just doing push to a remote repository.</p><p>🛠️ Let’s use the following command for this:</p><pre>git push https://gitlab.com/&lt;user name&gt;/gitlab-cd-react</pre><p>&lt;user name&gt; here is our GitLab.com username.</p><p>This will create a private project named gitlab-cd-react in your GitLab.com account.</p><p>🛠️ Please navigate to https://gitlab.com/&lt;user name&gt;/gitlab-cd-react.</p><p>After, we will set up the Kanban board and create several issues to have something to collect data about.</p><h4>🗃️ Creating issues and setting up a board</h4><p>Let’s start with creating an issue.</p><p>🛠️ Click <strong>Issues</strong> in the left side menu, then press one of the <strong>New Issue</strong> buttons in the same area. An issue “create” form will open. Specify “Create issues for the tutorial” as the title.</p><p>Specify the following text in description:</p><pre>Create issues with the titles listed below.<br>- [ ] Create labels<br>- [ ] Set up board<br>- [ ] Create continuous delivery pipeline<br>- [ ] Perform GitLab Flow iterations<br>- [ ] Study the metrics</pre><p>Yes, these are the things we are going to do. Leave default values for all the other fields. Press <strong>Submit issue</strong> button.</p><p>Please note that the list with items starting with square brackets was recognized as a checklist and its items as to-dos. GitLab, as many other platforms, uses <strong>Markdown</strong> for text formatting.</p><p>🛠️ Assign Create issues for the tutorial issue to yourself. To achieve this click <strong>Edit</strong> in the panel with text <strong>0 Assignees</strong> on the right in the issue &quot;edit&quot; form.</p><p>Congrats, now you are the assignee. Actually, the issue could have been assigned to any team member, but as nothing would really be different, we don’t spawn the unnecessary extra accounts.</p><p>🛠️ Now, let’s actually create all the issues listed in the description of the first one.</p><p>Only specify titles, leave default values for all the other fields. You can “check the boxes” in the Create issues for the tutorial issue description either as you create the other issues, or all at once when you are done with them.</p><p>There are different ways to create issues. When creating the issues from the list, find and use at least 3 different ways to do it. You should have 6 issues total in the end.</p><p>🛠️ Close Create issues for the tutorial issue. You can do this via <strong>Issues</strong> -&gt; <strong>List</strong> or <strong>Issues </strong>-&gt; <strong>Board</strong>. Click the issue&#39;s title and press <strong>Close issue</strong> button in the issue edit form.</p><h4>🎢 Our workflow</h4><p>Workflow is a sequence of stages a task passes to be considered “done”. In Continuous Delivery, Kanban or DevOps, the task moves through some sequence of stages forward or might be returned to one of the previous ones.</p><blockquote><em>This assumes linear value streams. You can </em><a href="https://en.wikipedia.org/wiki/Value-stream_mapping#Purpose_of_value-stream_mapping"><em>read about value streams here</em></a><em>. Reworking sometimes pretty baroque workflow diagram into a sequence of stages might be a tough management task, solving which is far beyond the scope of the article.</em></blockquote><p>We will use the simple stages’ sequence.</p><ul><li>Open</li><li>Dev</li><li>Dev: done</li><li>QA</li><li>Closed</li></ul><p>In the beginning, an issue is in <strong>Open</strong> stage. Then, it’s pulled into work by developers and moved to <strong>Dev</strong> stage. After the work is done, the issue is placed into the stage <strong>Dev: done</strong>.</p><blockquote><em>Why do we need the extra stage? This is because in </em><a href="https://en.wikipedia.org/wiki/Lean_manufacturing"><em>Lean</em></a><em>, which all the methodologies like CD, Kanban, and DevOps are based on, the next work center pulls work items when it’s ready as contrasted with the methodologies where work items are handled to the next work center by the previous. This makes it possible to discover work items waiting to be processed, so that work items accumulating before some stage reveal a bottleneck in our pipeline.</em></blockquote><p>Subsequently, the issue is pulled into <strong>QA</strong> stage where quality assurance and autotests do their job, and <strong>QA</strong> “graduates” are considered so good that they are deployed straight to production.</p><blockquote><em>“That’s not how it’s done, you know, that’s one-two-production, there have to be other environments like Staging in the process!” you might say. I totally agree, but here we make use of a pipeline as simple as possible both to implement and to understand. Otherwise, we might find ourselves burrowed under complexities originating, say, from Kubernetes.</em></blockquote><h4>🏷️ Setting up labels</h4><p>GitLab uses <strong>labels</strong> to organize issues. Labels are like tags and let you search, filter, and display issues in different places based on the existence of one or another label.</p><p>🛠️ Open the issue Create labels and assign it to yourself. Select <strong>Issues</strong> -&gt; <strong>Labels </strong>in the left menu, then click <strong>New label</strong> in the right part of the main area. Create so the following 3 labels picking colors you prefer.</p><ul><li>Dev</li><li>Dev: done</li><li>QA</li></ul><p>The names of the labels here are arbitrary and don’t carry any special sense from the GitLab’s perspective.</p><p>🛠️ When done, assign the status <strong>Closed</strong> to the issue Create labels.</p><p>Now everything is ready for setting up the Kanban board.</p><h4>📝 Setting up board</h4><p>GitLab has <a href="https://docs.gitlab.com/ee/user/project/issue_board.html">Kanban boards</a>. They help show different issues according to their labels in a meaningful way. Despite the name, the boards can be used not just to implement Kanban, but also Scrum, and other methodologies. We even already have an issue dedicated the boards.</p><p>🛠️ Assign the corresponding issue to yourself.</p><p>🛠️ Select <strong>Issues</strong> -&gt; <strong>Boards</strong> in the left menu. You’ll see the default board. It already have <strong>Open </strong>and <strong>Closed </strong>list columns. Press <strong>Add list</strong> in the top right part of the main page area. Create one list for each of the labels created earlier.</p><ul><li>Dev</li><li>Dev: done</li><li>QA</li></ul><p>If you created them in a different order, you can just drag the lists around by titles to change the order.</p><p>🛠️ On the <strong>Issues</strong> -&gt; <strong>Boards</strong> page, drag the Set up board issue from the <strong>Open</strong> column to the <strong>Dev</strong> column. Select the issue clicking the part of the card, not occupied with any text.</p><p>This way we indicate that we started working on the issue. Note that the label <strong>Dev</strong> has just been automatically assigned to the issue.</p><p>🛠️ As we in fact have already done all the work for the issue, move the issue to the <strong>Dev: done</strong> column.</p><p>We are closing to the stage <strong>QA</strong>. Try to impersonate a tester, think about how unreliable developers are, and, optionally, take a small appliance apart with a screwdriver.</p><p>🛠️ Then move the issue to <strong>QA</strong> stage. Here you can drag the issue from column to column back and forth several times. By the way, such adventures in fact eventually occur with issues at this stage. In the process, the label for a new column will appear and the label for the previous column will disappear for the issue. Finally, leave the issue in the <strong>Closed</strong> column and note that this action has closed it.</p><p>Another useful UI is <a href="https://gitlab.com/dashboard/todos">To-Do List</a>(there is a submenu new the login menu). This is the <a href="https://docs.gitlab.com/ee/user/todos.html">list of actions</a> expected from you for some reason. We will not dwell on it in detail, but I recommend checking it during the tutorial from time to time to get an idea of how it works.</p><h3>🚀 Continuous delivery pipeline</h3><p>In this section, we will implement Continuous Delivery using GitLab.</p><h4>🏗️ Implementing a CD pipeline using GitLab</h4><p>To actually run pipeline code, <strong>GitLab.com</strong> uses shared runners that implement <a href="https://docs.gitlab.com/runner/executors/">docker executors</a>. If you register your own agents, you have much more flexibility in <a href="https://docs.gitlab.com/runner/executors/">choosing build environments</a>.</p><p>Let’s now build our pipeline.</p><p>🛠️ Assign issue Create continuous delivery pipeline to yourself and transfer it into <strong>Dev</strong> stage.</p><p>🛠️ Let’s author the CI/CD pipeline. For this purpose, we need to create a file called .gitlab-ci.yaml. At the time of this writing, it could be done like this.</p><ol><li>Browse to <strong>gitlab-cd-react</strong> project.</li><li>Either on <strong>Project overview</strong> / <strong>Details</strong> or <strong>Repository</strong> page you will see files in your repository.</li><li>Select <strong>New file</strong> option as shown in the screenshot. You will be redirected to the file creation page.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1003/0*C0nteFD4-OvArfaV.png" /></figure><p>4. In the <strong>File name</strong> window, enter the name of the file .gitlab-ci.yml, the <strong>Template</strong> field to the right will be filled in automatically, another <strong>Apply a template</strong> field will appear. Check out the available options. Finally choose Bash - it&#39;s just a template with <strong>bash</strong> commands, we&#39;ll use that as a starting point.</p><p>Let’s take a look at the resulting file. The official document on the .gitlab-ci.yml file format is <a href="https://docs.gitlab.com/ee/ci/yaml/gitlab_ci_yaml.html">here</a>.</p><blockquote><em>The format that GitLab uses to define build pipelines is one of the simplest and most intuitive in the industry.</em></blockquote><pre>image: busybox:latest<br><br>before_script:<br>  - echo &quot;Before script section&quot;<br>  - echo &quot;For example you might run an update here or install a build dependency&quot;<br>  - echo &quot;Or perhaps you might print out some debugging details&quot;<br><br>after_script:<br>  - echo &quot;After script section&quot;<br>  - echo &quot;For example you might do some cleanup here&quot;<br><br>build1:<br>  stage: build<br>  script:<br>    - echo &quot;Do your build here&quot;<br><br>test1:<br>  stage: test<br>  script:<br>    - echo &quot;Do a test here&quot;<br>    - echo &quot;For example run a test suite&quot;<br><br>test2:<br>  stage: test<br>  script:<br>    - echo &quot;Do another parallel test here&quot;<br>    - echo &quot;For example run a lint test&quot;<br><br>deploy1:<br>  stage: deploy<br>  script:<br>    - echo &quot;Do your deploy here&quot;</pre><p>To get the pipeline we need, we slightly modify this file.</p><pre>image: busybox:latest</pre><p>image specifies which docker image will be used to run build jobs. GitLab uses Docker Hub by default, but this is configurable. We need an image with <strong>Node.js</strong> preinstalled.</p><p>🛠️ Use</p><pre>image: node:14-alpine</pre><p>before_script and after_script are executed before and after each job, respectively. In the process of running the jobs, we will use Node.js modules.</p><p>🛠️ We organize caching of modules in order not to download modules every time from the Internet.</p><pre>cache:<br>  key: ${CI_COMMIT_REF_SLUG}<br>  paths:<br>    - .npm/</pre><pre>before_script:<br>  - npm ci --cache .npm --prefer-offline</pre><ul><li>We first indicate where the jobs will look for the cache, namely — inside the current folder in the .npm subfolder. We use ${CI_COMMIT_REF_SLUG} as the cache key, which means we keep a separate cache for each Git branch.</li><li>In before_script, we install packages according to package-lock.json, specifying the .npm subfolder as the path for the cache.</li></ul><p>Thus, further in the .gitlab-ci.yml code, we can assume that the packages are already installed.</p><p>We will not need after_script in this tutorial.</p><p>Let’s move on to the build1 section, initially there should be something like this:</p><pre>build1:<br>  stage: build<br>  script:<br>    - echo &quot;Do your build here&quot;</pre><p>Let’s take a look at the stage: build chunk. stage defines the pipeline stage, the stages are executed one after another. One stage can contain one or more jobs that can be executed in parallel.</p><p>🛠️ Let’s change the commands to actually build something:</p><pre>build1:<br>  stage: build<br>  script:<br>  - npm run build<br>  artifacts:<br>    expire_in: 1 week<br>    paths:<br>    - &#39;build/&#39;</pre><ul><li>npm ci installs the required modules according to the package-lock.json file without checking if they already exist in the node_modules folder.</li><li>npm run build runs actual build of the minified React application.</li><li>artifacts specifies which files should be saved for future use. By default, <strong>webpack</strong> with the default <strong>create-react-app</strong> settings copies the files into build folder.</li><li>We use expires_in to avoid wasting space on builds that are no longer needed.</li></ul><p>Now let’s review the piece of code for the stage stage: test</p><pre>test1:<br>  stage: test<br>  script:<br>    - echo &quot;Do a test here&quot;<br>    - echo &quot;For example run a test suite&quot;</pre><pre>test2:<br>  stage: test<br>  script:<br>    - echo &quot;Do another parallel test here&quot;<br>    - echo &quot;For example run a lint test&quot;</pre><p>🛠️ We do not need the job test2, but we will change the job test1 as follows:</p><pre>test1:<br>  stage: test<br>  script:<br>  - &quot;CI=true npm test&quot; <br>  dependencies:<br>  - build1</pre><ul><li>npm test here runs the unit tests defined in our project.</li><li>dependencies indicates that the given job test1 depends on the results of the job build1 so that the artifacts of the job are available in this stage.</li></ul><p>Let’s move on to the part related directly to deployment.</p><pre>deploy1:<br>  stage: deploy<br>  script:<br>    - echo &quot;Do your deploy here&quot;</pre><p>This part is usually tied to the specifics of the environment in which the deployment is carried out and technically complex. GitLab supports deploying to Kubernetes with very little effort. The alternative is to implement the deployment logic by ourselves. We’ll leave the technical details of this process for future articles. The focus of this article is on explaining the basics of working with GitLab, so we will resort to a trick: we will use the fact that our React application is technically a static website and deploy it to GitLab Pages, which are available to anyone with an account on GitLab.com.</p><p>🛠️ Replace job deploy1 with this code.</p><pre>pages:<br>  stage: deploy<br>  script:<br>  - mv public _public<br>  - mv build public<br>  only:<br>  - master<br>  artifacts:<br>    paths:<br>    - public<br>  dependencies:<br>  - build1</pre><p>We renamed deploy1 to pages because GitLab, by the name of the job, understands that it is required to deploy the files available to this job to GitLab Pages. Next, we do 2 things.</p><ul><li>mv public _public saves the public folder generated from the <strong>create-react-app</strong> template. We do this because GitLab will return the contents of the folder named public in response to requests to Pages.</li><li>mv build public - here we just place the build result where the GitLab Pages web server will look for it.</li></ul><p>Let’s complete the work on our pipeline.</p><p>🛠️ Commit the edited file .gitlab-ci.yml, specify &quot;Add CI/CD pipeline&quot; as <strong>Commit message</strong>, and leave master as the branch.</p><p>🛠️ Move the Create continuous delivery pipeline issue to the <strong>Dev: done</strong> stage.</p><h3>🎠 A few rounds of work with GitLab Flow</h3><p>Git is a very flexible version control system and can be used in very different ways. If each team member uses Git in his own way, confusion arises when it is difficult to understand the logic of other people’s actions from the history of changes, and nobody understands what to expect from the others. Measuring performance isn’t even really possible in a situation like this. In order to avoid such confusion, team member usually agree on uniform rules for working with Git, and such an agreement is called Git workflow.</p><p>Let’s do the following to study the implementation of GitLab Flow in GitLab:</p><ul><li>discuss what Git workflow is in general;</li><li>see how access rights and protected branches are implemented in GitLab;</li><li>figure out how merge requests work;</li><li>do several iterations of GitLab Flow.</li></ul><h4>➡️ GitLab Flow</h4><p>In short, <a href="https://docs.gitlab.com/ee/topics/gitlab_flow.html">GitLab Flow</a> is:</p><ul><li>every time you need to change something in the code, a branch is created from the main (in Git, the default is master);</li><li>then a merge request is opened(called a pull request in other systems) where the changes are discussed;</li><li>there is a separate branch for each of the environments (test, production, etc.);</li><li>one of the “environment branches” from where the unconditional deployment takes place may coincide with the main one;</li><li>deployment occurs by merging into a branch of the corresponding environment;</li><li>merges happen from a feature branch to the main branch, or from a “less production” “environment branch” to a “more production” branch.</li></ul><blockquote><em>Actually, any Git workflow that supports trunk based development will suffice to implement CD, and we could implement any such workflow with GitLab. However, as GitLab recommends using it, and to keep things simple, we use GitLab Flow.</em></blockquote><h4>🔐 Access levels in GitLab</h4><p>There are different <a href="https://docs.gitlab.com/ee/user/permissions.html">access levels</a>, ordered from the highest to the lowest.</p><ul><li><strong>Instance administrator </strong>— available only for self-hosted installations, can do anything.</li><li><strong>Owner </strong>— the owner of a group of projects, can do anything except purely technical things such as enabling/disabling features and integrating with other services.</li><li><strong>Maintainer </strong>— can do anything on the project level and below except some actions related to the entire project, such as changing its name or visibility, as well as destructive actions such as deleting issues.</li><li><strong>Developer </strong>— can do anything <strong>Maintainer</strong> can, except some administrative and destructive actions inside projects, such as setting the branches protection and editing comments.</li><li><strong>Reporter </strong>— can edit issues, but cannot make changes to the repository.</li><li><strong>Guest </strong>— has read-only access to issues except confidential, and can create new issues.</li></ul><p>The permission levels are given here for completeness, we will not actively use them in the tutorial.</p><h4>🔀 Merge requests</h4><p><a href="https://docs.gitlab.com/ee/user/project/merge_requests/getting_started.html">Merge requests</a> are the main way to make code changes when using GitLab. Changes are made to the code, then a merge request is created by the author of the changes. The merge request is then discussed, the code review takes place, and, as a result, is accepted, sent for revision or rejected.</p><p>🛠️ Go to the issue Perform GitLab Flow iterations, assign it to yourself and move it to the <strong>Dev</strong> column.</p><p>🛠️ Move the task Create continuous delivery pipeline to <strong>QA</strong> stage.</p><blockquote><em>Yes, this is how we will test our pipeline, but you should approach testing more responsibly on real projects. Do as I say, not as I do 😉</em></blockquote><p>Our application is useless. This is not uncommon as there are many other useless applications on the Internet. To take it to the next level, let’s make our application toxic. There are 2 features that are proven to help with this.</p><p>🛠️ Open src/App.js file for editing (<strong>Repository</strong> -&gt; <strong>Files</strong>, <strong>Edit</strong>) and add this line right at the top of the file.</p><pre>import {useEffect} from &#39;react&#39;;</pre><p>For <strong>Commit message</strong> specify, for example, “Add React imports”. Leave the <strong>Target branch</strong> as master and hit <strong>Commit changes</strong>.</p><p>Oops, we just committed the changes directly to master. Some workflows allow this, but we will comply with GitLab Flow. A GitLab feature called <strong>protected branches</strong> will help us to avoid such incidents in the future. Its purpose is not to control access, but to help team members work within the agreements and to avoid accidentally changing or deleting data in the repository.</p><p>🛠️ Click in the left menu <strong>Settings</strong> -&gt; <strong>Repository</strong>. In the page that opens, find the <strong>Protected Branches</strong> section.</p><p>You will see that only the master branch is protected by default, but you can add others. By default</p><ul><li><strong>force push</strong> is forbidden to all;</li><li><strong>push </strong>allowed for Maintainers;</li><li><strong>merge </strong>allowed for Maintainers.</li></ul><p>🛠️ Change <strong>Allowed to push</strong> to <strong>No one</strong>.</p><p>🛠️ Great, let’s go back to editing src/App.js. After the line</p><pre>function App() {</pre><p>and before the line</p><pre>  return (</pre><p>insert</p><pre>  useEffect(() =&gt; {<br>    alert(&#39;Consent to cookies and everything!&#39;);<br>  }, []);</pre><p>The corresponding piece of code should now look like this</p><pre>// ...other code<br>function App() {<br>  useEffect(() =&gt; {<br>    alert(&#39;Consent to cookies and everything!&#39;);<br>  }, []); </pre><pre>  return (<br>// ...other code</pre><p>Specify “Add the annoying popup” as the commit message.</p><p>You will see that you are prompted to add a commit to a certain branch and a certain branch name has already been automatically generated. We will change this name in the future to something more meaningful and consistent with our Git workflow, but for now, let’s do a little experiment.</p><p>🛠️ Replace the suggested name with master and click <strong>Commit changes</strong>.</p><p>You will see the error message You are not allowed to push into this branch. Good! Branch protection works. Let&#39;s now initiate the procedure for making changes to the code as it is supposed to be done.</p><p>🛠️ Now change the branch name from master to feature-cookies-consent. Leave the S<strong>tart a new merge request with these changes</strong> option checked. A branch with the specified name will be created and the changes will be committed to that branch. You will be taken to the merge request creation page. Leave</p><ul><li>the default merge request title.</li><li>Unassigned for <strong>Assignee</strong> and <strong>Reviewer</strong>.</li><li>everything else as default.</li></ul><p>Click <strong>Submit merge request</strong></p><p>By the way</p><ul><li><strong>Assignee</strong> is responsible for working on the merge request and merging it into the target branch,</li><li><strong>Reviewer</strong> examines the proposed changes and can approve them if they look correct.</li></ul><p>After you submit the merge request, you will be taken to its page. Let’s explore it. In the main area, we see the following.</p><ul><li>Which branch we are merging to which (feature-cookies-consent to master).</li><li>The pipeline status for the feature-cookies-consent branch.</li><li>Ability to approve the merge request on behalf of the current user.</li><li>The <strong>Merge</strong> button to accept the merge request and the merging of the branches, which results in changes in master.</li><li>The area where you can see the commits that will be merged; you can change the merge commit message there.</li><li>Ways to take part in the discussion: express your attitude and leave a smile or add a comment.</li><li>The <strong>Close merge request</strong> button, which allows you to close the merge request without accepting changes to the master code.</li></ul><blockquote><em>For free subscriptions there is only the possibility of “optional” approval of a merge request, in paid versions it is possible to make the approval mandatory.</em></blockquote><p>If you switch to the <strong>Changes</strong> tab, you can see what changes are planned to the code. Here you can also create a comment that will refer to a line of code.</p><p>Merge requests also support labels that you can use to make them easier to find.</p><p>By this time, the pipeline should have finished and… the tests have failed. In our particular case, this is because we used window.alert, which is a browser object and our unit tests running in Node.js environment do not have access to it.</p><p>For Continuous Delivery, a good set of automated tests is required because we rely on them for <strong>QA</strong> almost completely, and completely in the case of Continuous Deployment. Keeping such a test suite up to date is the main technical challenge when implementing such methodologies.</p><p>🛠️ Let’s fix src/App.js by adding a check that the code runs in the browser. Let&#39;s put the code that uses window.alert inside the code block that verifies that we are in the browser. Make sure you use feature-cookies-consent when you edit the file. It should look something like this</p><pre>// ...other code<br>function App() {<br>  useEffect(() =&gt; {<br>    if (typeof process === &#39;undefined&#39; || process.release === undefined) {<br>      alert(&#39;Consent to cookies and everything!&#39;);<br>    }<br>  }, []); </pre><pre>  return (<br>// ...remaining code</pre><p>Add “Ensure running in a browser” as the commit message. Target branch should be feature-cookies-consent.</p><p>🛠️ Let’s go back to the merge request Add the annoying popup and click the <strong>Merge</strong> button and commit the changes to the code. Leave the <strong>Delete source branch</strong> checkbox checked.</p><p>When using GitLab Flow, feature branches are traditionally removed after merge. This allows to avoid cluttering the system with no longer relevant branches and makes it possible create a branch with the same name in case of an issue being sent for revision.</p><p>🛠️ Open <strong>Repository</strong> -&gt; <strong>Graph</strong> and notice that feature-cookies-consent is now merged into master. Then open the src/App.js file in the master branch and notice that the code now contains our edits.</p><p>🛠️ Go to the issue Perform GitLab Flow iterations and click the down arrow next to the <strong>Create merge request</strong> button in the main area. Specify the name of the branch you want to create in order to base a new merge request on it. feature-notifications-consent is fine. Leave the source branch as master.</p><p>When creating a merge request like this, the issue will be closed automatically as soon as we merge the code into the main branch.</p><p>Note that the auto-generated title of our merge request starts with Draft:. This means that <a href="https://docs.gitlab.com/ee/user/project/merge_requests/drafts.html">merge request is marked as draft</a>. This is useful for authors to indicate that the work is in progress and to avoid merging the changes &quot;as is&quot; as a result of misunderstanding. The same can be achieved with the <strong>Mark as draft</strong> button in the upper right part of the main area.</p><p>Let’s add push notifications to our website to make it look modern.</p><p>🛠️ Let’s edit the code slightly differently for a change. On the merge request page, click the <strong>Open in Web IDE</strong> button. An interface that is slightly more convenient for working with code will open. From there, open the src/App.js. Add the code</p><pre>      Notification.requestPermission().then(function(result) {<br>        alert(`You ${result} notifications`);<br>      });</pre><p>before the line which we added earlier</p><pre>      alert(&#39;Consent to cookies and everything!&#39;);</pre><p>🛠️ Click the <strong>Commit</strong> button at the bottom left. You will see the commit interface, specify the commit message “Add the notifications users want”. Leave everything else as default and hit the <strong>Commit</strong> button.</p><p>🛠️ Go to merge request, for example, using the submenu at the top right of the screen next to the login menu. Click the <strong>Mark as ready</strong> button at the top right of the merge request form. Click the <strong>Merge</strong> or <strong>Merge when pipeline succeeds</strong> button depending on whether the pipeline has already completed.</p><p>🛠️ Open our website and see if the new “features” work and our page feels like most modern sites on the Internet. Normally, GitLab Pages serve pages from https://&lt;user name&gt;.gitlab.io/gitlab-cd-react where &lt;user name&gt; is your GitLab.com username.</p><p>🛠️ If everything is fine, drag Create continuous delivery pipeline issue to the <strong>Closed</strong> stage in the board.</p><p>So, we went through several full work cycles, and now we can study the metrics that GitLab collected in the process.</p><h3>📈 CI/CD Metrics in GitLab</h3><p>During teamwork, it is useful to collect statistics to understand whether the productivity is increasing or decreasing. The way to measure performance can be very different depending on the nature of work and the type of project or product.</p><blockquote><em>For example, a web studio may have “typical” tasks such as coding a page layout as a part of a standard package of services offered to clients. An agile startup may be at the stage when the product vision is changing with a growing understanding of the customer and the market, and tasks can be difficult to predict and often unique. Still, we can come up with purely technical performance indicators such as time to market(TTM) or just the duration of a particular stage. Tracking the dynamics of such indicators can be very useful: over a long enough period of time, this allows you to understand how the performance is changing.</em></blockquote><p>The good news is that GitLab has the ability to collect the statistics. Let’s check out the functionality.</p><p>🛠️ Assign Study the metrics issue to yourself and drag it to <strong>Dev</strong> stage.</p><p>🛠️ Click on <strong>Analytics</strong>. By default, the <a href="https://docs.gitlab.com/ee/user/analytics/value_stream_analytics.html"><strong>Value stream</strong> section</a> will open. In Lean in general, and in DevOps in particular, it is believed that tasks ultimately bring some benefit to the customer. And movement of such useful tasks, from the stage of inception through all stages of work to the moment when the results become available to the customer, is called the value stream.</p><p>Main average values ​​by stage:</p><ul><li><strong>Issue </strong>— the time it takes to “take issue into work” i.e. label or add to a Milestone.</li><li><strong>Plan </strong>— the time from the last action in the previous stage till the first commit appears in the branch, at least one of the commits of which is associated with the same issue. That is, “the time it takes to start committing”.</li><li><strong>Code </strong>— the lifetime of a branch associated with a particular issue, which passes before the merge request appears.</li><li><strong>Test </strong>— the time from the start to the end of all pipelines for this project.</li><li><strong>Review </strong>— the time from the creation of the merge request to its merge or closure.</li><li><strong>Staging </strong>— the time from accepting a merge request to deploying to a production environment.</li></ul><p>If you add up the duration of <strong>Issue</strong>, <strong>Plan</strong>, <strong>Code</strong>, <strong>Review</strong>, and <strong>Staging</strong>, you get roughly time to market (TTM).</p><p>At the top of the page, you will also see some aggregate indicators for the project for the selected period of time.</p><p><strong>Analytics</strong> -&gt; <strong>Repository</strong> shows different graphs related to commit languages, code coverage metrics(if configured), distribution of commits over time(month, days of the week, hours).</p><p><strong>Analytics</strong> -&gt; <strong>CI/CD</strong> shows summary data on executed pipelines, including a graph showing the dynamics of changes in the pipelines “successful” executions.</p><p>🛠️ Close Study the metrics issue.</p><h3>🎉 Congratulations!</h3><p>You’ve walked through implementing CD with GitLab starting from issues and boards and finishing with the metrics.</p><p>If you noticed any problem with the course or know how to improve it, please do create an issue <a href="https://github.com/ntaranov/gitlab-cd-react">in the course starter repository</a>.</p><h3>Links</h3><ol><li><a href="https://about.gitlab.com/handbook/product/gitlab-the-product">GitLab product handbook</a></li><li><a href="https://www.amazon.ca/DevOps-Handbook-World-Class-Reliability-Organizations/dp/1942788002">The DevOps Handbook</a></li><li><a href="https://git-scm.com/book/en/v2">Pro Git Book</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=74872d5e2d96" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Common Continuous Integration Team Scenarios]]></title>
            <link>https://redpill-solutions.medium.com/common-continuous-integration-team-scenarios-7fee96a7136b?source=rss-93445ab8e6cf------2</link>
            <guid isPermaLink="false">https://medium.com/p/7fee96a7136b</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[git]]></category>
            <category><![CDATA[github]]></category>
            <category><![CDATA[continuous-delivery]]></category>
            <category><![CDATA[continuous-integration]]></category>
            <dc:creator><![CDATA[Nick Taranov | Fractional CTO]]></dc:creator>
            <pubDate>Mon, 10 Aug 2020 18:35:02 GMT</pubDate>
            <atom:updated>2020-09-08T21:56:49.403Z</atom:updated>
            <content:encoded><![CDATA[<blockquote><em>Read </em><a href="https://github.com/ntaranov/continuous-integration-team-scenarios-text/blob/master/article.md"><em>the article on GitHub</em></a><em> for better formatting</em></blockquote><p>You learned Git commands but want to know how real-world Continuous Integration is done? Or maybe you already doing it but want to fine-tune your everyday actions? This course will give you practical skills of doing Continuous Integration (CI) in a GitHub-based repository. The course isn’t intended to be a click-through wizard-style experience, instead you’ll do what developers do during their daily work the way they do it. I’ll explain the theory as you reach relevant steps.</p><p><em>Below is the course walkthrough. It is not required to complete the course. You can refer to the walkthrough when in trouble. I explain main points and screencast doing all the course tasks in the video.</em></p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2Ft_7l9hxwLis%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Dt_7l9hxwLis&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/cdfa41f8c51fa0da2ef9c6f1d35ed295/href">https://medium.com/media/cdfa41f8c51fa0da2ef9c6f1d35ed295/href</a></iframe><h3>💻 What are we going to do?</h3><p>As we progress, we will incrementally build a memo with a common CI steps sequence, which is a great way to memorize it. In other words, we will make the list of actions which developers take doing CI doing CI. We’ll also use some simple test suite to mimic actual CI process.</p><p>Here is a GIF showing commits in your repository as you will go through the course. As you see, nothing here is complicated and everything is essential.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/0*2QMgI9z3NBgohsDi.gif" /></figure><p>You’ll go through these common CI scenarios:</p><ul><li>Working on a feature;</li><li>Using auto tests to ensure quality;</li><li>Implementing a bugfix;</li><li>Resolving merge conflicts;</li><li>Dealing with a bug in production.</li></ul><h3>📖 What you’ll learn</h3><p>You’ll be able to answer the questions:</p><ul><li>What is Continuous Integration, really?</li><li>What types of auto tests are used with CI and when they are triggered?</li><li>What are pull requests and when you need them?</li><li>What is Test Driven Development and how it relates to CI?</li><li>To merge or to rebase?</li><li>To roll back or to fix forward?</li></ul><h3>❔ What is Continuous Integration, really?</h3><p><strong>Continuous integration</strong>, or CI, is a technical practice when each member of a team integrates his code at least once a day so that the resulting code at least builds without errors.</p><p>There are various interpretations of the term. One point of disagreement is the frequency of integration. Some argue that integrating just once a day is not enough to actually integrate continuously. A counterexample is when everybody gets fresh code once in the morning and integrates once in the evening. Though the argument stands, it’s mostly considered that “once a day” definition is practical enough, it’s concrete and is good for teams of different sizes.</p><p>Another point is that C++ in not the only language used in development for a long time, and just building without errors is a very weak validation requirement. Some set of tests (e.g. unit tests run on a local computer) should be required to pass. Currently, community seems to gravitate towards such requirement, and in the future “build + units tests” will likely be the mainstream, if not already.</p><p>Continuous integration is different from continuous deployment or <strong>continuous delivery (CD)</strong> in that we are not required to have a release candidate after each integration cycle.</p><h4>This is the list of CI steps we will use during the course</h4><ol><li>Pull in the latest code. Create a branch from master. Start working.</li><li>Create commits on your new branch. Build and test locally. Pass? Go to the next step. Fail? Fix errors or tests and try again.</li><li>Push to your remote repository or remote branch.</li><li>Open a pull request. Discuss the changes, add more commits as discussion continues. Make tests pass on the feature branch.</li><li>Merge/rebase commits from master. Make tests pass on the merge result.</li><li>Deploy from the feature branch to production.</li><li>If everything is good in production for some period of time, merge changes to master.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/706/0*71yH1KgG85YrIMm4.png" /></figure><h3>⚙️ Before you begin</h3><h4>Have the software in place</h4><p>To complete the course you will need <a href="https://nodejs.org/">Node.js</a> and a <a href="https://git-scm.com/downloads">Git client</a> installed.</p><blockquote><em>👉 You can use whatever Git client you prefer, but I will provide reference solutions for command line only.</em></blockquote><p>Make sure, you have a command line client. If you don’t have a command line client yet, you can find the installation instructions <a href="https://git-scm.com/downloads">here</a>.</p><h4>Setup the repository</h4><p>You will need to fork <a href="https://github.com/ntaranov/continuous-integration-team-scenarios-students">the course starter repository</a> on GitHub. Let’s agree to call your fork of the repository <strong>course repository</strong>.</p><p>Done? If you went with the default configuration, you will likely now have a repository named continuous-integration-team-scenarios-students under your GitHub account, and the URL probably looks like</p><pre>https://github.com/&lt;your GitHub user name&gt;/continuous-integration-team-scenarios-students</pre><p>I’ll just refer to it as &lt;repository URL&gt; from now on.</p><blockquote><em>👉 I will use angle brackets like </em><em>&lt;this&gt; when you need to substitute the expressions with whatever values relevant for you.</em></blockquote><p>Make sure that <strong>GitHub actions</strong> are enabled for the course repository. If actions are not enabled, please enable them clicking the button in middle of the screen under the Actions GitHub UI tab.</p><blockquote><em>👉 You will not be able follow the instructions in the course if GitHub Actions are not enabled.</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/663/0*Bq2n9nzmtm5GRXNo.png" /></figure><p>You can always use GitHub’s ability to render Markdown to see the current state of the list we compose at</p><pre>https://github.com/&lt;your GitHub user name&gt;/continuous-integration-team-scenarios-students/blob/master/ci.md</pre><h4>About solutions</h4><p>Even though the best way to complete this course is doing everything by your own hands, you potentially might find yourself in trouble.</p><p>If you feel that you don’t understand what to do and cannot continue, you can refer to solution branch included in your starting repository. Please, don&#39;t merge solution into master during the course. You can use the branch to figure out what you need to do, or to compare your code with the reference implementation using all the capabilities Git provides. If you feel completely lost, you can replace your master branch commits with solution commits entirely and then reset your working directory to whatever step of the course you desire.</p><blockquote><em>👉 Use it only if you really need to. Skip to the next section if you don’t need the information now.</em></blockquote><p>Commit your current work before doing this.</p><pre>git add .<br>git commit -m &quot;Backing up my work&quot;</pre><p>The following commands</p><ul><li>rename master as master-backup;</li><li>rename solution as master;</li><li>check out the new master and overwrite the working directory;</li><li>create branch “solution” from “master” (being “solution” in past) just is in case if need “solution” in future.</li></ul><pre>git branch -m master master-backup </pre><pre>git branch -m solution master </pre><pre>git checkout master -f </pre><pre>git branch solution</pre><p>After performing these actions, you can use git log master to figure out what commit you need. Then you can reset your working directory to it using</p><pre>git reset --hard &lt;the SHA you need&gt;</pre><p>If you are happy with the result, eventually you might want to push the remote. Don’t forget to update the remote branch accordingly when you push.</p><pre>git push --force origin master</pre><p>Please, note that we use git push --force. You will rarely ever want to force push to a remote&#39;s master, but here we have very specific scenario with only one collaborator who knows what he is doing.</p><h3>🏁 Starting working</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/0*Mcm8YhSloUPHK809.gif" /></figure><p>Let’s start composing our CI steps list. You would commonly start this step with pulling the latest code from the remote repository, but we don’t have a local repository set up yet, so we clone it instead.</p><h4>⌨️ Activity: Pull in the latest code, create a branch from master, start working</h4><ol><li>Clone the course repository from &lt;repository URL&gt;.</li><li>Run npm install in the course repository&#39;s directory; we need it to install Jest which we use to run tests.</li><li>Create a branch and name it feature. Check out the branch.</li><li>Add the tests code to ci.test.js between the comments asking to do so.</li></ol><pre>  it(&#39;1. pull latest code&#39;, () =&gt; {<br>    expect(/.*pull.*/ig.test(fileContents)).toBe(true);<br>  });<br><br>  it(&#39;2. add commits&#39;, () =&gt; {<br>    expect(/.*commit.*/ig.test(fileContents)).toBe(true);<br>  });<br><br>  it(&#39;3. push to the remote branch with the same name&#39;, () =&gt; {<br>    expect(/.*push.*/ig.test(fileContents)).toBe(true);<br>  });<br><br>  it(&#39;4. open a pull request and continue working&#39;, () =&gt; {<br>    expect(/.*pull\s+request.*/ig.test(fileContents)).toBe(true);<br>  });</pre><p>5. Add the text with the first 4 steps to “ci.md” file.</p><pre>1. Pull in the latest code. Create a branch from `master`. Start working.    <br>2. Create commits on your new branch. Build and test locally.  <br>  Pass? Go to the next step. Fail? Fix errors or tests and try again.  <br>3. Push to your remote repository or remote branch.  <br>4. Open a pull request. Discuss the changes, add more commits  <br>  as discussion continues. Make tests pass on the feature branch.</pre><h4>Commands</h4><pre># Clone the course repository<br>git clone &lt;repository URL&gt;<br>cd &lt;repository name&gt;<br><br># Run npm install in the course repository&#39;s directory; we need it to install Jest which we use to run tests.<br>npm install<br><br># Create a branch and name it feature. Check out the branch.<br>git checkout -b feature<br><br># Edit ci.test.js as described above<br># Edit ci.md as described above</pre><h3>🔨 Create commits on your new branch, build and test locally</h3><p>We are going to set up tests to run on commit and then commit the code.</p><h4>Scenarios when tests are triggered automatically</h4><p>Locally:</p><ul><li>Constantly or in response to relevant code changes with immediate feedback;</li><li>On save (more often for interpreted or JIT compiled languages);</li><li>On build (more often when compilation is required);</li><li>On commit;</li><li>On push.</li></ul><p>On the build server or build environment:</p><ul><li>When code is pushed to a private branch / repository.</li></ul><ol><li>The branch code is tested.</li><li>The potential merge result (usually with master) is tested.</li></ol><ul><li>In a stage of Continuous Integration / Continuous Delivery pipeline</li></ul><p>Generally, the less time it takes to run a test suite, the more often you can afford to run it. Common arrangement might look like this.</p><ul><li>Fast unit tests — on build, in CI pipeline.</li><li>Slow unit tests, fast component and integration tests — on commit, in CI pipeline.</li><li>Slow component and integration tests — in CI pipeline.</li><li>Security, capacity and other lengthy or expensive tests — in CI/CD pipelines, but only on certain build modes/stages/pipelines, e.g., when preparing a release candidate or when launched manually.</li></ul><h4>⌨️ Activity</h4><p>I suggest first running tests manually using npm test command. After, let&#39;s add a git hook to run our tests on commit. There is a catch, Git hooks are not considered a part of repository and thus cannot be cloned from GitHub with the rest of the course materials. To install the hook, you&#39;ll need to run install_hook.sh or copy file repo/hooks/pre-commit into your local .git/hooks/ directory. When you commit, you&#39;ll see test running and checking if certain keywords are present in the list.</p><ol><li>Run tests manually executing npm test command in your course repository folder. Make sure test run.</li><li>Install the pre-commit hook executing install_hook.sh.</li><li>Commit changes to local repository.</li><li>Make sure the tests run before the commit.</li></ol><p>You repository should look like this after the actions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*6-j6jrnZh_pLDTpgYe8Lmw.gif" /></figure><h4>Commands</h4><pre># Install the pre-commit hook executing install_hook.sh.  <br><br># Commit changes to local repository. Use &quot;Add first CI steps&quot; as the commit message.<br>git add ci.md ci.test.js<br>git commit -m &quot;Add first CI steps&quot;<br><br># Make sure the tests run before the commit.</pre><h3>⏫ Push to your remote repository or remote branch</h3><p>After doing work locally, developers usually make their work publicly available so that it could be eventually merged into a shared codebase. Using GitHub, it’s either done publishing the work to a personal fork or a branch.</p><ul><li>When using forks, a developer forks a remote shared repository creating a personal remote copy of it also known as fork. After, he clones the personal repo to work with it locally. When work is done and the commits are created, he pushes them to his fork where they are available to others and can be merged into the shared repository. This approach commonly used when contributing to open source projects on GitHub. It is also used in my extended <a href="https://devops.redpill.solutions/">Team Work and CI with Git</a> course.</li><li>Another approach is to use single remote and to consider only shared remote’s master branch &quot;protected&quot;. In this scenario individual developers push their work into the remote&#39;s branches so that it could be reviewed and, if everything is good, merged to the remote&#39;s master.</li></ul><p>In this specific course, we will use the branch-based workflow.</p><p>Let’s now publish our work.</p><h4>Commands</h4><pre>git push --set-upstream origin feature</pre><h3>👀 Open a pull request.</h3><p>Open a pull request named <strong>Steps review</strong>. Set feature as the head branch and master branch of your forked repository as the base branch.</p><blockquote><em>👉 Make sure that you set </em><em>master in your </em><strong><em>forked repository</em></strong><em> as base, I will not answer to pull requests to the course starter repository.</em></blockquote><p>In GitHub slang, “base” branch is the branch you base your work on, and “head” branch is the branch with the work you are merging.</p><h3>💬 Discuss the changes, add more commits as discussion continues</h3><h4>Pull requests</h4><p>Pull request(PR) is a way to discuss, review and document code. Pull requests are named after a common way of integrating individual contributions into shared codebase. Normally, a person clones a remote official project repository and works on the code locally. After, he pushes the code into his personal remote repository and asks the official repository’s maintainers to <strong>pull</strong> his code into their local repositories where they review and potentially <strong>merge</strong> it. The same concept is known under different names, e.g., “merge request”.</p><p>Actually, you don’t have to use pull request feature of GitHub or similar platforms. Teams can use other modes of communication including face-to-face communication, voice calls, or e-mails, but there is a number of reasons to use such forum thread-like pull requests. Here are some of them:</p><ul><li>to organize discussions related to specific changes in code;</li><li>as a place to show feedback on work in progress, from both auto tests and colleagues;</li><li>to formalize code reviews;</li><li>to make it possible to figure out the reasons and considerations behind a given piece of code later.</li></ul><p>You generally open a pull request when you need to discuss something or receive feedback. For example, if you work on a feature that might be implemented in a number of ways, you might want to open a pull request even before writing the first line of code to share your ideas and to discuss your plans with the collaborators. If the work is more straightforward, the pull request is opened when something is already done, committed, and can be discussed. In some scenarios, you might open a PR just for the sake of quality assurance: to trigger auto tests or to initiate a code review. Whatever you decide, don’t forget to @mention persons whose approval is required in your pull request.</p><p>Usually, you do the following when creating a PR.</p><ol><li>Specify what you suggest to merge where.</li><li>Write a description explaining the purpose of the pull request. You might want to:</li></ol><ul><li>add anything important that isn’t obvious from the code and anything helpful to understand the context such as relevant #issues and commit numbers;</li><li>@mention everyone you want to start collaborating with, or you can @mention them in a comment later;</li><li>ask your colleagues to do or check something specific.</li></ul><p>After you open a PR, whatever automated tests scheduled are run on your code. In our case, this is going to be the same test suite as we ran locally, but in a real project, there might be extra tests and checks.</p><p>Please, wait until the tests finish running. You can see the status of the tests at the bottom of the PR discussion thread. Proceed when the tests are done.</p><h3>⚠️ Add remark about the CI steps being opinionated</h3><p>The list used in this course is opinionated, we should add a remark stating this.</p><h4>⌨️ Activity: Create a pull request for the remark</h4><ol><li>Check out master branch.</li><li>Create a branch named bugfix.</li><li>Add the remark text at the bottom of ci.md.</li></ol><pre>&gt; **GitHub flow** is sometimes used as a nickname to refer to a<br>  flavor of trunk-based development when code is deployed straight<br>  from feature branches. This list is just an interpretation<br>  that I use in my [DevOps courses](<a href="http://redpill.solutions/">http://redpill.solutions</a>).<br>  The official tutorial is [here](<a href="https://guides.github.com/introduction/flow/">https://guides.github.com/introduction/flow/</a>).</pre><p>4. Commit the changes.</p><p>5. Push branch bugfix to the remote.</p><p>6. Open a pull request named <strong>Adding a remark</strong> with head branch bugfix and base branch master.</p><blockquote><em>👉 Make sure that you set </em><em>master in your </em><strong><em>forked repository</em></strong><em> as base, I will not answer to pull requests to the course starter repository.</em></blockquote><p>This is what your repository should look like.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*Jr1LlfC2_kRv8TYToB32rA.gif" /></figure><h4>Commands</h4><pre># Check out master branch. Create a branch named bugfix.<br>git checkout master<br><br># Create a branch named bugfix-remark.<br>git checkout -b bugfix<br><br># Add the remark text at the bottom of ci.md.<br><br># Commit the changes<br>git add ci.md<br>git commit -m &quot;Add a remark about the list being opinionated&quot;<br><br># Push branch bugfix to the remote.<br>git push --set-upstream origin bugfix<br><br># Open a pull request using GitHub UI as described above</pre><h3>🔀 Merge the “Adding a remark” pull request</h3><h4>⌨️ Activity</h4><ol><li>Open the pull request.</li><li>Click “Merge pull request”</li><li>Click “Confirm merge”</li><li>Click “Delete branch” as we no longer need it</li></ol><p>This is the commits diagram after the merge.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*dkYqkCalUNqbPdPRG127FQ.gif" /></figure><h3>▶️ Continue working and adding tests</h3><p>Collaborating on a pull request often results in additional work being requested. Usually, this is a result of a review or a discussion, but for our course, we are going to simulate this by adding another item to our CI checklist.</p><p>Some test coverage is usually in place to support Continuous Integration. The tests coverage requirements differ and are usually found in a document like “contribution guidelines”. We are going to be straightforward and will add a test for each line in our checklist.</p><p>When working through the activity part, first try to commit tests. If you installed the pre-commit hook correctly earlier, the test we&#39;ve just added will fail and nothing will be committed. Please note that that this is how we know that our tests actually check something. Curiously, if we started with the code before tests, tests passing could mean either that the code works as expected or that the tests don&#39;t really check anything. Also, if we didn&#39;t write tests first, we could forget writing them completely, as nothing would remind us about it.</p><h4>Test Driven Development (TDD)</h4><p>TDD advocates writing tests before code. Usual TDD work process looks like this.</p><ol><li>Add a test.</li><li>Run all tests and see if the new test fails.</li><li>Write the code.</li><li>Run tests.</li><li>Refactor code.</li><li>Repeat.</li></ol><p>Because not passing tests are usually displayed in red, and passing displayed in green, the cycle is also known as “red-green-refactor”.</p><h4>⌨️ Activity</h4><p>First try to commit the tests and let them fail, then add and commit the actual checklist text. You will see that tests are passing (“green”). Then, push the new code to the remote and see how tests are run in the pull request and the pull request’s status is updated.</p><ol><li>Checkout branch feature.</li><li>Add the following tests to ci.test.js after the last it(...); call.</li></ol><pre>  it(&#39;5. Merge/rebase commits from master. Make tests pass on the merge result.&#39;, () =&gt; {<br>    expect(/.*merge.*commits.*tests\s+pass.*/ig.test(fileContents)).toBe(true);<br>  });<br><br>  it(&#39;6. Deploy from the feature branch to production.&#39;, () =&gt; {<br>    expect(/.*Deploy.*to\s+production.*/ig.test(fileContents)).toBe(true);<br>  });<br><br>  it(&#39;7. If everything is good in production for some period of time, merge changes to master.&#39;, () =&gt; {<br>    expect(/.*merge.*to\s+master.*/ig.test(fileContents)).toBe(true);<br>  });</pre><p>3. Try to commit the tests. If the pre-commit hook is in place, the commit will fail.</p><p>4. After, add this text to ci.md.</p><pre>5. Merge/rebase commits from master. Make tests pass on the merge result.  <br>6. Deploy from the feature branch with a sneaky bug to production.<br>7. If everything is good in production for some period of time, merge changes to master.</pre><p>5. Add and commit the changes locally.</p><p>6. Push the changes to feature branch.</p><p>Now you should have something like this</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*7EuKsGF70n2dTFYrlvuE3Q.gif" /></figure><h4>Commands</h4><pre># Checkout branch feature<br>git checkout feature<br><br># Add tests to ci.test.js as described above<br><br># we add ci.test.js to commit it later<br>git add ci.test.js<br><br># Try to commit the tests. If the pre-commit hook is in place, the commit will fail.<br>git commit<br><br># After, add text to ci.md as described above<br><br># Add and commit the changes locally<br>git add ci.md<br>git commit -m &quot;Add the remaining CI steps&quot;<br><br># Push the changes to feature branch.<br>git push</pre><h3>🚫 Merge Conflict</h3><p>Navigate to <strong>Steps review</strong> pull request.</p><p>Even though we didn’t do anything wrong, and tests pass for our code, we still cannot merge branch feature into master. This is because another branch bugfix was merged into master as we were working on the pull request. This creates situation when remote master branch has commits newer than the one we based branch feature on. Because of this, we cannot just add commits from feature to master and fast-forward master HEAD to the tip of feature. In this situation we need to either merge or rebase feature on master. GitHub can actually perform automatic merge if there is no conflicts. Alas, in our situation both branches have concurrent changes to file ci.md, a situation known as merge conflict, and we need to resolve it manually. To make history compatible with the remote master branch, we can both merge feature into master or rebase it on master.</p><h4>To Merge or to Rebase</h4><h4>Merge</h4><ul><li>Creates an extra merge commit and preserves work history</li></ul><ol><li>Contains initial head branch commits with original timestamps and authors</li><li>Preserves commit SHAs and references to them in pull requests discussion</li></ol><ul><li>Requires resolving conflicts just once</li><li>Makes history non-linear</li></ul><ol><li>History might be hard to read because of lots of branches (think IDE cable)</li><li>Makes automatic debugging harder, e.g., makes git bisect less useful - it&#39;s only going to find a merge commit</li></ol><h4>Rebase</h4><ul><li>Replays commits from head branch on top of base branch one-by-one</li></ul><ol><li>New commits with new SHAs are formed resulting in the commits being linked to original pull requests but not to the relevant comments</li><li>Commits might be recombined and amended in the process or squashed into fewer, even into just one commit</li></ol><ul><li>Might require resolving a number of conflicts</li><li>Supports linear history</li></ul><ol><li>Might be easier to read if not too long without a good reason</li><li>Auto debugging and troubleshooting is somewhat easier: supports git bisect, might make automatic rollbacks more granular and predictable</li></ol><ul><li>Requires force pushing the rebased head branch when used with pull requests</li></ul><p>Usually, teams agree to employ the same strategy anytime they need to merge changes. It can be “pure” merge or rebase or somethings in-between like doing interactive rebase locally for the branches not published to remote, but merges for “public” branches.</p><p>We’ll use merge here.</p><h4>⌨️ Activity</h4><ol><li>Make sure latest code is pulled to local master branch.</li><li>Check out branch feature.</li><li>Merge master branch into it. A merge conflict related to concurrent changes to ci.md will be reported.</li><li>Resolve the conflict so that both our CI list and the remark about it remain.</li><li>Push the merge commit into remote feature.</li><li>Check the pull request status in GitHub UI, wait until the merge is permitted.</li></ol><h4>Commands</h4><pre># Make sure latest code is pulled to local `master` branch<br>git checkout master<br>git pull<br><br># Check out branch feature<br>git checkout feature<br><br># Merge master branch into it<br>git merge master<br><br># A merge conflict related to concurrent changes to ci.md will be reported<br># =&gt; Auto-merging ci.md<br>#    CONFLICT (content): Merge conflict in ci.md<br>#    Automatic merge failed; fix conflicts and then commit the result.<br><br># Resolve the conflict so that both our CI list and the remark about it remain.<br># edit ci.md so that it doesn&#39;t contain the merge confilict markers<br>git add ci.md<br>git merge --continue<br># you can leave the default commit message<br><br># Push the merge commit into remote feature.<br>git push<br><br># Check the pull request status in GitHub UI, wait until the merge is permitted.</pre><h3>👍 Good job!</h3><p>You finished working on the list and now you need to merge the pull request into master.</p><h4>⌨️ Activity: 🔀 Merge the “Steps review” pull request</h4><ol><li>Open the pull request.</li><li>Click “Merge pull request”</li><li>Click “Confirm merge”</li><li>Click “Delete branch” as we no longer need it</li></ol><p>This is your repo now</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*XMfdXGxN3hNwv9Y417RTiA.gif" /></figure><h3>💥 An Error was Found in Production</h3><p>They say “testing can be used to show the presence of bugs, but never to show their absence.” Even though we had some tests and they didn’t show us any bugs, a sneaky bug made it into production.</p><p>In a scenario like this we need to take care of:</p><ul><li>whatever deployed into production;</li><li>the code in master branch with the bug, which might be used to start new work.</li></ul><h4>Rolling Back vs Fixing Forward</h4><p>“Rolling back” is deploying a known good previous version to production and reverting the code changes containing bug. “Fixing forward” is adding the fix to master and deploying a new version as soon as possible. With continuous delivery and good test suite in place, because APIs and database schemes change as code deployed to production production, rolling back is generally much harder and riskier than fixing forward.</p><p>Because reverting does not bring any risk at all for us, we are going the “revert” way to simulate situation when we</p><ul><li>fix the production ASAP;</li><li>make the code in master immediately suitable for starting new work.</li></ul><h4>⌨️ Activity</h4><ol><li>Checkout master branch locally.</li><li>Pull the latest changes form the remote.</li><li>Revert the merge commit of <strong>Steps review</strong> PR in master.</li><li>Push to remote.</li></ol><p>This is the history with the merge reverted</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*69uSe92Am_XojO9htqBgOg.gif" /></figure><h4>Commands</h4><pre># Checkout master branch locally.<br>git checkout master<br><br># Pull the latest changes form the remote.<br>git pull<br><br># Revert the merge commit of Steps review PR in master.<br># We are reverting merge commit, so we need to indicate which line we choose to remain<br>git show HEAD<br><br># let&#39;s suppose, the previous master branch tip commit listed first by the previous command<br>git revert HEAD -m 1<br># you can leave the standard commit message<br><br># Push to remote.<br>git push</pre><h4>☝️ Self-check</h4><p>Make sure ci.md no longer contains the text &quot;sneaky bug&quot; after you revert the merge commit.</p><h3>Fix CI list and return it to master</h3><p>We reverted feature merge entirely. Good news is that now we don&#39;t have the bug in master. Bad news is that our precious continuous integration list is gone too. So, ideally we need to apply fix over commits from feature and return them to master along with the fix.</p><p>We can approach the task differently:</p><ul><li>revert the commit which reverts the merge of feature to master;</li><li>cherry-pick commits from the former feature.</li></ul><p>Different teams use different approaches, and we will cherry-pick the commits into a separate branch and will create a separate pull request for the new branch.</p><h4>⌨️ Activity</h4><ol><li>Create a branch called feature-fix and checkout it.</li><li>Cherry-pick all the commits of the former feature into the new branch. Resolve the merge conflicts.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/0*qqW1zczO5xFNVfVk.gif" /></figure><p>3. Add the regression test to ci.test.js:</p><pre>it(&#39;does not contain the sneaky bug&#39;, () =&gt; {<br>  expect( /.*sneaky\s+bug.*/gi.test(fileContents)).toBe(false);<br>});</pre><p>4. Run tests locally to make sure they don’t pass.</p><p>5. Delete text “ with a sneaky bug” in ci.md.</p><p>6. Add the changes to tests and to the steps list and commit them.</p><p>7. Push the branch to remote.</p><p>You should have something similar as the result</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*n21XY5lxM_CG0yOSCDmwBw.gif" /></figure><h4>Commands</h4><pre># Create a branch called feature-fix and checkout it.<br>git checkout -b feature-fix<br><br># Cherry-pick all the commits of the former feature into the new branch.<br># use log to figure out hashes for the commits<br># - preceding one adding the first list items: C0<br># - adding the last items to list: C2<br>git log --oneline --graph<br>git cherry-pick C0..C2<br># resolve the merge conflicts like you did merging feature-steps into master<br># - edit ci.md and/or ci.test.js<br># - add the files to index<br># - run &quot;git cherry-pick --continue&quot;, you can leave the commit message<br><br># Add the regression test to ci.test.js<br># Run tests locally to make sure they don&#39;t pass.<br><br># Delete text &quot; with a sneaky bug&quot; in ci.md.<br><br># Add the changes to tests and to the steps list and commit them.<br>git add ci.md ci.test.js<br>git commit -m &quot;Fix the bug in steps list&quot;<br><br># Push the branch to remote.<br>git push --set-upstream origin feature-fix</pre><h3>Open a pull request</h3><p>Open a pull request named <strong>Fixing the feature</strong>. Set feature-fix as the head branch and master as the base branch. Please, wait until the tests finish running. You can see the status of the tests at the bottom of the PR discussion thread.</p><blockquote><em>👉 Make sure that you set </em><em>master in your </em><strong><em>forked repository</em></strong><em> as base, I will not answer to pull requests to the course starter repository.</em></blockquote><h3>🔀 Merge the “Fixing the feature” pull request</h3><p>Thanks for the great fix! Please go ahead and merge the request to master.</p><h4>⌨️ Activity</h4><ol><li>Click “Merge pull request”</li><li>Click “Confirm merge”</li><li>Click “Delete branch” as we no longer need it</li></ol><p>This is what you should have at the point</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*iNIDctDQmfIVpOlaDb2bug.gif" /></figure><h3>🎉 Congratulations!</h3><p>You performed all the actions which people commonly do during Continuous Integration.</p><p>If you noticed any problem with the course or know how to improve it, please do create an issue in the <a href="https://github.com/ntaranov/continuous-integration-team-scenarios-students">course starter repository</a>. This course also have <a href="https://lab.github.com/ntaranov/common-continuous-integration-team-scenarios">an interactive version</a> using GitHub Learning Lab as a platform.</p><blockquote><em>Please share the course with your friends if you found it useful.</em></blockquote><blockquote><em>The course is licensed under </em><a href="https://github.com/ntaranov/continuous-integration-team-scenarios-students/blob/master/LICENSE"><em>CC-BY-4.0</em></a><em> © Nick Taranov</em></blockquote><p><em>Originally published at </em><a href="https://github.com/ntaranov/continuous-integration-team-scenarios-text/blob/master/article.md"><em>https://github.com</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7fee96a7136b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Common DevOps Myths Managers Believe]]></title>
            <link>https://redpill-solutions.medium.com/common-devops-myths-managers-believe-f4767c1d46c3?source=rss-93445ab8e6cf------2</link>
            <guid isPermaLink="false">https://medium.com/p/f4767c1d46c3</guid>
            <category><![CDATA[myths]]></category>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[management]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[product-management]]></category>
            <dc:creator><![CDATA[Nick Taranov | Fractional CTO]]></dc:creator>
            <pubDate>Tue, 21 Apr 2020 04:47:45 GMT</pubDate>
            <atom:updated>2020-04-21T04:47:45.326Z</atom:updated>
            <content:encoded><![CDATA[<p>There is the DevOps handbook, the most authoritative source. It has an entire section dedicated to myths about DevOps. Why write an article? Well, in my opinion, the book not only explains the methodology, but also sells it. This is why the myth section only debunks misconceptions, which the authors picked for the purpose, and not the ones actually popular. Here I’m intending to bust the myths circulating in the real world which I stumbled upon explaining the methodology to my listeners. If you prefer videos, here it is. Please, scroll down for the text.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2F3pffQZhyLCI%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D3pffQZhyLCI&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2F3pffQZhyLCI%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/22d32b34c9fd9b8848e96ffa80d388f5/href">https://medium.com/media/22d32b34c9fd9b8848e96ffa80d388f5/href</a></iframe><h3>1. DevOps Applies to You</h3><p>No, DevOps is not for everyone. The myth comes first because even some of the founding fathers seem to deliberately encourage it. For example, Gene Kim et al. in The DevOps Handbook somehow forgets to mention that DevOps is not for everyone, but speculates on its universal applicability. Check out the quote (I put the questionable parts in <em>italics</em>): “… there is now an overwhelming weight of evidence that the problems described above happen <em>almost everywhere</em>, and that the <em>solutions associated with DevOps are nearly universally applicable.</em>” Here is more: “In <em>almost every</em> IT organization, there is an inherent conflict between Development and IT Operations which creates a downward spiral …”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/881/1*uM78lHAPz7usAhA_FFAQLw.png" /><figcaption>These two are necessary for DevOps</figcaption></figure><p>I beg to differ, there are a whole lot of IT organizations for which it’s not the case at all, or in part, or even is not applicable. Some organizations do not have an IT operations department or a production environment, for example, consulting shops. They often have a customer support department to help with SLAs, but they may not have one at all e.g. companies that customize Wordpress, or consultants who are involved in security testing. There are IT organizations without a software development department. Many hosting companies are such organizations. They might have quite complicated tech arrangements, but all the software can be either someone else’s “box”, or “cloud”, or written by external consultants.</p><p>John Allspaw, writes more carefully (but John, why do you write this?): “<em>No matter what industry you are in, or what product or service your organization provides</em>, this way of thinking is paramount and necessary for survival <em>for every business</em> and technology leader.”</p><p>Sorry to hurt your consulting guys, but it’s not. Do you believe that DevOps as a way of thinking is the paramount in building bridges and tunnels? Then I have a bridge to sell you.</p><h3>2. DevOps Will Make Your Life Easier</h3><p>Let’s suppose that DevOps can possibly be implemented for your business function at least to some extent. However, DevOps might be no good for you. There ain’t no such thing as a free lunch:</p><ul><li>implementing DevOps will require restructuring your organization, this might be a long and risky undertaking;</li><li>it will be necessary to invest resources to rework your technical practices;</li><li>and, most importantly, continuous delivery will require additional costs for each new feature, every day.</li></ul><p>The difference in difficulty of implementing DevOps for different software delivery models is massive. It is not at all difficult to implement DevOps for a web project in a small company, whereas just DevOps technical bits of Chrome browser development ecosystem are huge and monstrously complex.</p><p>It may be that the very nature of your business does not imply benefiting from DevOps. For example, you release computer games with hefty marketing budgets. You can quickly release a new version after problems with 1.0.0. Will your users tolerate the problems and keep from uninstalling the game until 1.0.1 arrives? Or maybe you are writing software for a nuclear reactor or a massive industrial system where the cost of error is just too high. Continuous delivery does not make practical sense where it’s not feasible to tolerate an eventual error in production.</p><p>To put it short, if such benefits as</p><ul><li>short time to market (TTM),</li><li>fast automated low-risk deployments,</li><li>an up-to-date release candidate available at any given moment,</li><li>steady and predictable flow of work</li></ul><p>are not enough to justify paying the price, then there is no point in implementing DevOps for you.</p><h3>3. DevOps is for Internet Companies Only</h3><p>One of the most seminal reports “10 Deploys a Day” by John Allspaw and Paul Hammond indeed was about the needs of Flickr, an online service. The most famous DevOps success stories arguably happened with online companies.</p><p>However, if you have a store, you don’t have to switch the entire business to DevOps, you can only do this with the online department. If you have a bank, you do not have to transfer all the business functions to the new methodology, but you will likely get an advantage if you employ DevOps for the web user interfaces development.</p><p>Currently, there are lots of success stories of “non-online” companies that have transferred parts of their business functions to DevOps.</p><h3>4. DevOps is a Technical Thing</h3><p>There is an opinion that DevOps is something that software developers, admins, and other “technical people” do. Yes, it is difficult to imagine DevOps without the technical part properly implemented, but the gist of DevOps movement is about people.</p><p>Even ultra-fast deployment will not dramatically reduce Time to Market without reworking processes into end-to-end streams of tasks, without sharing knowledge and responsibility, and agreements with business and within teams.</p><p>In order to solve the problem of building quality in, it’s not enough to just write a lot of tests and collect a bunch of telemetry. All these measures should be deeply understood by each member of the team responsible for the implementation, and, ideally, should come from them. All this should be done understanding why, where and how much you need it.</p><p>Without scientific approach to work, measuring results and testing hypotheses, it is difficult to imagine the very efficiency and reliability that DevOps can deliver.</p><p>Note that the above is not purely technical, but rather the opposite. And, as in the well-known <a href="http://catb.org/jargon/html/koans.html#id3141171">koan</a>, actions without understanding will give nothing.</p><h3>5. Implementing DevOps Is Following a Process or Using a Tool</h3><p>Is it possible to follow some kind of flowchart or install some DevOps Server Ultimate 9000 and just do what the tool says?</p><p>For simplicity, I’ll start with the tool. Yes, implementing DevOps completely without automation is unrealistic, but in most cases you can do with Linux, git, and bash. Will it be as effective as possible? Often not. Will it work? Yes. From DevOps point of view, same task can be solved in completely different ways. Sometimes you can use Docker, but usually you can solve the same problem with a fixed farm, Puppet and deploying with git, or you can opt to implement the same thing in the cloud using vendor tools. If you prefer, use Jenkins, or Bamboo, or, say, TeamCity, and, in any case, there are GNU Make and shell scripts. On the contrary, purchasing or installing all the tools and waiting for a miracle to happen is the same as buying an expensive tracksuit, the best running shoes in the world and waiting for Olympic gold to be brought to you.</p><p>It’s a bit more complex about the process. Let’s talk about SCRUM. SCRUM is a rather formalized Agile, which suits some teams and doesn’t suit the others. A number of teams still could use a different version of Agile, taking into account the specifics of the teams and the businesses. In this sense DevOps is more like Agile in a broad sense than specifically SCRUM. This is why there is no single process or even a set of exact requirements for DevOps. In addition, the production model for DevOps is Lean. With Lean, the same thing is true: you can apply a set of approaches, but you don’t have a one-size-fits-all step-by step approach with a common endpoint. Such business, such Lean. The same is true for DevOps.</p><h3>6. DevOps Specialists are the Only Specialists Required</h3><p>Even assuming that such “broad specialists” exist, many areas of expertise require years to master and then demand constant practice to stay current: network engineers, programmers, QA specialists, etc.<br>In addition, some tasks are best done with different mindsets. We all know that testing by developers only and having dedicated testers are completely different things.</p><p>Involving different specialists in joint effort to help them deliver as much as possible using their knowledge and experience is what DevOps advocates, and not renaming one of occupation roles, for example, admins, into DevOps engineers.</p><h3>7. DevOps Replaces Existing Product Management Approaches and Tools</h3><p>New consultant — new methodology — new life? Not at all necessary. Firstly, maybe DevOps just doesn’t suit you. Secondly, DevOps is not something started from scratch. Yes, DevOps is an umbrella term that describes a number of managerial and technical approaches used together. But most of these approaches are not new at all, e.g. they were used in Toyota, GM and Ford factories.</p><p>For example, if you already have Lean implemented and use the “end-to-end” Kanban for all tasks, starting with gathering requirements and ending with launching in the production environment, then maybe you already have 50 percent of DevOps If you also have CD, then it’s rather 80 percent.</p><p>Development team uses SCRUM with 1 week cycle with great results? No need to break anything. You absolutely can do with a weekly deployment cycle, this is legal. You can have a weekly development cycle and time to market (TTM) of 1 week, or you might be able to deploy in 17.6 seconds, and have the development cycle and TTM of 2 months.</p><p>It may seem that continuous delivery has fundamental contradictions with good practices in IT, security and the like. The suspicions are not entirely unfounded, and DevOps definitely has its limits of applicability. However, you can try to partition your systems so that compliance is isolated in parts that are changed rarely. If the compliance is internal, maybe the time has come to stop rubber stamping, and design realistic approval procedures?</p><p>In any case, the entirety of the knowledge you have, and not just DevOps, will help you to better organize work.</p><h3>There Are Others</h3><p>This concludes my short review of the myths about DevOps that circulate in reality. Next time I will describe the common misconceptions which have some merit to them.</p><p>What myths or deliberately imposed delusions about DevOps have you encountered?</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f4767c1d46c3" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>