<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Squadcast - Medium]]></title>
        <description><![CDATA[Squadcast - Medium]]></description>
        <link>https://medium.com/squadcast?source=rss----19ae28e78429---4</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>Squadcast - Medium</title>
            <link>https://medium.com/squadcast?source=rss----19ae28e78429---4</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 15:57:50 GMT</lastBuildDate>
        <atom:link href="https://medium.com/feed/squadcast" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Improving Customer Support with Squadcast Webforms: A Smart Solution for MSPs]]></title>
            <link>https://medium.com/squadcast/improving-customer-support-with-squadcast-webforms-a-smart-solution-for-msps-2fd2c936d2c9?source=rss----19ae28e78429---4</link>
            <guid isPermaLink="false">https://medium.com/p/2fd2c936d2c9</guid>
            <category><![CDATA[on-call]]></category>
            <category><![CDATA[best-practices]]></category>
            <category><![CDATA[software-update]]></category>
            <dc:creator><![CDATA[Squadcast]]></dc:creator>
            <pubDate>Thu, 09 May 2024 06:36:20 GMT</pubDate>
            <atom:updated>2024-05-09T06:36:20.685Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/855/0*FsgY-j4jNaUjX174.png" /></figure><p>Managed Service Providers (MSPs) handle a multitude of customer support cases, each requiring efficient routing to the right team member. Squadcast’s Webforms provide a solution to expedite issue reporting and streamline resolution. In this blog, we will explore how MSPs can leverage webforms to enhance the customer support experience.</p><h3>Common MSP Use Case</h3><ol><li><strong>Hotline Numbers:</strong> Traditionally, using hotline numbers poses challenges such as missed calls and delays, leading to extended resolution times for IT support requests. Moreover, direct engagement in calls by <a href="https://support.squadcast.com/schedules/create-and-manage-on-call-schedules-and-rotations">On-Call </a>members can be time-consuming and may not efficiently capture all necessary details.</li><li><strong>Streamlining Internal IT Support:</strong> Internally, webforms can streamline IT support requests for employees within the organization. Employees can use these webforms to report technical issues, ensuring timely assistance and improving organizational efficiency.</li></ol><h3>Solution: Adopting Squadcast Webforms</h3><p>To overcome the limitations of hotline numbers, MSPs can adopt <a href="https://support.squadcast.com/webforms/webforms">Squadcast Webforms</a>, which offer several key advantages:</p><ul><li><strong>Structured </strong><a href="https://www.squadcast.com/incident-response"><strong>Incident Reporting</strong></a>: Webforms provide a standardized method for customers to report issues, ensuring all essential details are captured accurately.</li><li><strong>Efficient </strong><a href="https://www.squadcast.com/"><strong>Incident Management</strong></a>: When customers fill out Webforms on the MSP’s website or support portal, incidents are automatically created in Squadcast, setting off the regular on-call roster and escalation process.</li><li><strong>Seamless Communication</strong>: Webforms supports notifications for customers who reported the incident, keeping them informed of the incident’s acknowledgement and resolution.</li><li><strong>Reduced Manual Routing</strong>: With <strong>predefined services and </strong><a href="https://www.youtube.com/watch?v=LE3yEZgUHDY"><strong>routing rules</strong></a>, incidents are directly routed to the subject matter experts, minimizing the time and effort spent on manual routing within the organization.</li><li><strong>Customizable and Professional</strong>: Squadcast Webforms are customizable, allowing MSPs and other organizations to add their company logo and tailor input fields for different types of incidents.</li></ul><h3>Implementing Squadcast Webforms: The How-To Guide</h3><p><a href="https://www.squadcast.com/blog/introducing-webforms-live-call-routing-incident-management-process"><strong>Creating multiple Webforms</strong></a> with Squadcast is a breeze. MSPs can customize forms <strong>based on specific customer needs</strong>, such as regional impact or affected services. Every Webform has a<strong> unique URL</strong> that can be easily <strong>embedded in customer accounts</strong> or <strong>product portals</strong>, providing a seamless incident reporting experience.</p><p>Additionally, Squadcast also offers the option for a <strong>custom domain.</strong> This means that you can use your own domain name, such as <a href="https://support.example.com,/">https://support.example.com,</a> for your Webforms instead of the default Squadcast domain.</p><p>Here are the steps to configure Webforms in Squadcast:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*RM992pl0JxU2jHlL.png" /></figure><p><strong>1. Form Configuration, </strong>where you can add a name &amp; check your form public URL or check the checkbox to set up a custom domain instead. You can also add tags here. Tags on webforms help classify different webforms better.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Urhbu5NW6AomeED1.png" /></figure><p><strong>2. Owner &amp; Services, </strong>where you can add an owner that enables streamlined assignment of incidents and service requests from customers. You can add services for which you’re creating these webforms.</p><p>Once the form is submitted, an incident is automatically created in Squadcast. This <strong>also triggers the escalation policy associated with particular services</strong>, notifying subject matter experts based on the services being affected.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Fnu8ZWGdrMzXW-UZ.png" /></figure><p><strong>3. Input Configuration, </strong>allows adding custom input fields for webforms like severity, priority, etc. which will help users to define their problems more clearly.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*J_mHIxicFmCOjVgc.png" /></figure><p><strong>4. Additional Details, </strong>which includes editing Form Header, Title, and Description fields, customizing your logo, etc. You can choose to <strong>send email updates to the reporter for the following stages of the reported issue: trigger, acknowledged by internal team, and resolution.</strong></p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2Fy-hNub20UnE%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Dy-hNub20UnE&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2Fy-hNub20UnE%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/26e13bb7b2b3a76c2c5a6cb296e7a988/href">https://medium.com/media/26e13bb7b2b3a76c2c5a6cb296e7a988/href</a></iframe><h3>Avoiding Spam on Webforms</h3><p>One of the primary concerns when utilizing webforms for incident reporting is the<strong> potential for spam submissions. </strong>These unsolicited and irrelevant reports can overwhelm the incident management system, leading to wasted time and resources. To address this challenge and ensure that only legitimate client incidents are being recorded, a strategic approach is required.</p><p>The proposed solution<strong> tailors Webforms for Client Validation</strong>. This approach safeguards the incident reporting process while making it convenient for clients to report issues.</p><h4>Suppression and Discard Rules</h4><p>Two effective strategies stand out in managing incidents through webforms while minimizing spam:</p><p><strong>(a) Suppression Rule: Filtering Out Unwanted Incidents</strong></p><p>By implementing suppression rules, incidents originating from email addresses outside of the client domain are classified as suppressed. While these incidents are still logged in the incident management system, they are marked as suppressed and do not trigger notifications. This approach strikes a balance between logging all incidents for reference while preventing unnecessary notifications for non-client reports.</p><p><strong><em>Read More: </em></strong><a href="https://www.squadcast.com/blog/keeping-stakeholders-notified-of-incidents-with-squadcast"><strong><em>Keeping Stakeholders Notified of Incidents With Squadcast</em></strong></a></p><p><strong>(b) Merge Suppression with Discard Function: Minimizing Noise</strong></p><p>For a more rigorous approach, combining the suppression rule with a discard function ensures that incidents not meeting the client domain criteria are not logged in the incident management system. This step minimizes noise by preventing the creation of irrelevant incidents altogether, sparing valuable time and effort that would otherwise be spent sifting through spam reports.</p><p><strong><em>Read More: </em></strong><a href="https://www.squadcast.com/case-studies/kovai-case-study"><strong><em>Kovai Reduced MTTA (Mean time to acknowledge) by 55% with Squadcast</em></strong></a></p><figure><a href="https://register.squadcast.com/"><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*x4h5Iu5tTItPJvRMjhSIrg.png" /></a></figure><h3>Enabling Smooth Incident Reporting for Enhanced Customer Support</h3><p>Squadcast Webforms offer <a href="https://www.squadcast.com/blog/msps-as-nocs-handling-multiple-clients">MSPs </a>a highly efficient alternative to traditional hotline numbers, streamlining incident reporting and enhancing the incident management process. By using Webforms, MSPs can achieve faster issue resolution, improved customer satisfaction, and seamless communication with customers.</p><p>As they continue to evolve in delivering top-notch support, <a href="https://support.squadcast.com/webforms/webforms">Squadcast Webforms</a> serve as a valuable tool, enabling them to handle incidents swiftly and effectively reducing your <a href="https://www.squadcast.com/case-studies/resolve-technology-case-study">MTTA </a>&amp; <a href="https://www.squadcast.com/blog/how-squadcast-actions-help-you-reduce-mttr">MTTR</a>. They can elevate your incident management capabilities and enhance customer support while also acting as a potential assistance to live On-Call Routing.</p><p><a href="https://www.squadcast.com/"><em>Squadcast</em></a><em> is an Incident Management tool that’s purpose-built for SRE. Get rid of unwanted alerts, receive relevant notifications and integrate with popular ChatOps tools. Work in collaboration using virtual incident war rooms and use automation to eliminate toil.</em></p><figure><a href="https://register.squadcast.com/"><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*RnT8Oam3vwVhvNls.png" /></a></figure><p><em>Originally published at </em><a href="https://www.squadcast.com/blog/improving-customer-support-with-squadcast-webforms-a-smart-solution-for-msps"><em>https://www.squadcast.com</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2fd2c936d2c9" width="1" height="1" alt=""><hr><p><a href="https://medium.com/squadcast/improving-customer-support-with-squadcast-webforms-a-smart-solution-for-msps-2fd2c936d2c9">Improving Customer Support with Squadcast Webforms: A Smart Solution for MSPs</a> was originally published in <a href="https://medium.com/squadcast">Squadcast</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[RapidSpike + Squadcast: Routing Alerts Made Easy]]></title>
            <link>https://medium.com/squadcast/rapidspike-squadcast-routing-alerts-made-easy-43803ff10e1d?source=rss----19ae28e78429---4</link>
            <guid isPermaLink="false">https://medium.com/p/43803ff10e1d</guid>
            <category><![CDATA[integration]]></category>
            <category><![CDATA[incident-management]]></category>
            <dc:creator><![CDATA[Squadcast]]></dc:creator>
            <pubDate>Thu, 28 Mar 2024 10:06:27 GMT</pubDate>
            <atom:updated>2024-03-28T10:06:27.347Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*v2Qd6AYAooWdLWiZ.png" /></figure><p>​ <a href="https://www.rapidspike.com/">RapidSpike</a> is a website monitoring solution that focuses on all three key aspects of website health: performance, reliability and security in a single dashboard.</p><p>If you use RapidSpike for your website monitoring requirements, you can integrate it with <a href="https://www.squadcast.com/">Squadcast</a>, an end-to-end <a href="https://www.squadcast.com/blog/traditional-vs-modern-incident-response">Incident Response</a> tool, to route alerts from RapidSpike to the right users in Squadcast with ease.</p><h3>Using RapidSpike as an Alert Source</h3><p><strong>Step1:</strong> In <strong>Squadcast</strong>, navigate to <strong>Services</strong> and then to <strong>Service Overview</strong>. Next, select or search for your Service. Expand the accordion and in the <strong>Alert Sources</strong> section, click <strong>Add</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Hq72KfPXnK0FUOE-.png" /></figure><p><strong>Step2: </strong>Select <strong>RapidSpike.</strong> Copy the displayed <strong>Webhook URL</strong> to<a href="https://support.squadcast.com/integrations/alert-source-integrations-native/rapidspike#create-a-squadcast-webhook-alert-in-rapidspike"> configure</a> it within <strong>RapidSpike.</strong> Finish by clicking <strong>Add Alert Source</strong> and <strong>Done.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Yt4z7-7cWKOOpy-M.png" /></figure><p><strong>Please Note: </strong>When an alert source turns Active, it’ll show up under Configured Alert Sources. You can either generate a test alert from the integration or wait for a real-time alert to be generated by the Alert Source. An Alert Source is active if there is a recorded incident via that Alert Source for the Service.</p><h3>Create a Squadcast Webhook Alert in RapidSpike</h3><p><strong>Step1: </strong>Login to your <strong>RapidSpike</strong> dashboard and navigate to the <strong>Settings</strong> menu. Under the <strong>Settings</strong> menu, click on <strong>Integrations</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*wEHTI7UVT41zJIuE.png" /></figure><p><strong>Step2: </strong>Under the <strong>Integrations</strong> menu, click on <strong>Webhooks</strong>. Then click on <strong>+ Add</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*cz0NbXxmwx8W7egF.png" /></figure><p><strong>Step3: </strong>Fill in the <strong>Label</strong>, paste the previously copied Squadcast Webhook URL in the placeholder for <strong>Webhook URL</strong> and select <strong>Request Type</strong> as <strong>POST</strong>. Then click on <strong>Add.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*JhkangryIngWm7Do.png" /></figure><p>That’s it, you are good to go! Your RapidSpike integration is now complete. Whenever RapidSpike fires an alert, an incident will be created in Squadcast for it.</p><p><a href="https://www.rapidspike.com/">RapidSpike</a> is a website monitoring solution that focuses on all three key aspects of website health. Integrating RapidSpike with Squadcast can help your organization in <a href="https://www.squadcast.com/blog/reduce-toil-with-better-alerting-systems">routing detailed alerts and enhance your Incident Management</a> process.‍</p><p><a href="https://www.squadcast.com/"><em>Squadcast</em></a><em> is a Reliability Workflow platform that integrates On-Call alerting and Incident Management along with SRE workflows in one offering. Designed for a zero-friction setup, ease of use and clean UI, it helps developers, SREs and On-Call teams proactively respond to outages and create a culture of learning and continuous improvement.</em></p><figure><a href="https://register.squadcast.com/"><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*bgiCvKlsAoIHYpTg.png" /></a></figure><p><em>Originally published at </em><a href="https://www.squadcast.com/blog/rapidspike-squadcast-routing-alerts-made-easy"><em>https://www.squadcast.com</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=43803ff10e1d" width="1" height="1" alt=""><hr><p><a href="https://medium.com/squadcast/rapidspike-squadcast-routing-alerts-made-easy-43803ff10e1d">RapidSpike + Squadcast: Routing Alerts Made Easy</a> was originally published in <a href="https://medium.com/squadcast">Squadcast</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Postmark + Squadcast Integration: Simplifying Alert Routing]]></title>
            <link>https://medium.com/squadcast/postmark-squadcast-integration-simplifying-alert-routing-6ee08b5ada96?source=rss----19ae28e78429---4</link>
            <guid isPermaLink="false">https://medium.com/p/6ee08b5ada96</guid>
            <category><![CDATA[best-practices]]></category>
            <category><![CDATA[incident-management]]></category>
            <category><![CDATA[integration]]></category>
            <dc:creator><![CDATA[Squadcast]]></dc:creator>
            <pubDate>Wed, 13 Mar 2024 14:15:25 GMT</pubDate>
            <atom:updated>2024-03-13T14:15:24.964Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*jqCItDfzRWpSM43Z.png" /></figure><p>​ <a href="https://postmarkapp.com/">Postmark</a> is a simple email delivery system used to send transactional and marketing emails and it ensures getting them delivered to the inbox on time, every time. It also helps in reducing email delivery time considerably.</p><p>If you use Postmark for your email delivery requirements, you can integrate it with <a href="https://www.squadcast.com/">Squadcast</a>, a Reliability Workflow Platform, to <a href="https://support.squadcast.com/services/alert-routing">route detailed alerts</a> from Postmark to the right users in Squadcast.</p><p>The below steps will help you set up Postmark and Squadcast integration.</p><h3>Using Postmark as an Alert Source</h3><p><strong>Step1:</strong> Navigate to <strong>Services</strong>, in <strong>Service Overview</strong>, select or search for your Service. Expand the accordion and in the Alert Sources section, click on <strong>Add</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*6eC_2I7aVCJ2XcLO.png" /></figure><p><strong>Step2:</strong> Select <strong>Postmark.</strong> Copy the displayed <strong>Webhook URL</strong> to<a href="https://support.squadcast.com/integrations/alert-source-integrations-native/postmark#create-a-squadcast-webhook-alert-in-postmark"> configure</a> it within <strong>Postmark.</strong> Finish by clicking <strong>Add Alert Source</strong> and click on <strong>Done.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*DUHOiSa_-REDWm_Z.png" /></figure><p><strong>Please Note:</strong> When an alert source turns Active, it’ll show up under Configured Alert Sources, you can either generate a test alert from the integration or wait for a real-time alert to be generated by the Alert Source. An Alert Source is active if there is a recorded incident via that Alert Source for the Service.</p><h3>Create a Squadcast Webhook Alert in Postmark</h3><p><strong>Step1:</strong> Login to your <strong>Postmark dashboard</strong>. Under the <strong>Servers</strong> tab, select <strong>Message Streams</strong> and click on the desired <strong>Message Stream</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ZJrG7ITn_8epxkEt.png" /></figure><p><strong>Step2: </strong>Under the <strong>Webhooks</strong> tab, click on <strong>Add webhook.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Foua0GUaE3sTcEzR.png" /></figure><p><strong>Step3: </strong>Now, paste the previously copied Squadcast Webhook URL in the <strong>Webhook URL</strong> placeholder. Select the desired events and click on <strong>Save webhook.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/558/0*_J5gX-ng2q0QESHy.png" /></figure><p>That’s it, you are good to go! Your Postmark-Squadcast integration is now complete. Whenever Postmark fires an alert, an incident will be created in Squadcast for it. <a href="https://postmarkapp.com/">‍</a></p><p><a href="https://postmarkapp.com/">Postmark</a> is a popular email delivery service used by various organizations to deliver transactional emails efficiently. Integrating Postmark with Squadcast can help your organization in routing detailed alerts and enhance your <a href="https://www.squadcast.com/blog/best-practices-in-incident-management">Incident Management practices</a>.‍</p><p><a href="https://www.squadcast.com/"><em>Squadcast</em></a><em> is an incident management tool that’s purpose-built for SRE. Get rid of unwanted alerts, receive relevant notifications and integrate with popular ChatOps tools. Work in collaboration using virtual incident war rooms and use automation to eliminate toil.</em></p><figure><a href="https://register.squadcast.com/"><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*QK7CtSSrxzpZCQEG.png" /></a></figure><p><em>Originally published at </em><a href="https://www.squadcast.com/blog/postmark-squadcast-integration-simplifying-alert-routing"><em>https://www.squadcast.com</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6ee08b5ada96" width="1" height="1" alt=""><hr><p><a href="https://medium.com/squadcast/postmark-squadcast-integration-simplifying-alert-routing-6ee08b5ada96">Postmark + Squadcast Integration: Simplifying Alert Routing</a> was originally published in <a href="https://medium.com/squadcast">Squadcast</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Hiteshwar shares his thoughts on being an SRE]]></title>
            <link>https://medium.com/squadcast/hiteshwar-shares-his-thoughts-on-being-an-sre-9ac9a8b987db?source=rss----19ae28e78429---4</link>
            <guid isPermaLink="false">https://medium.com/p/9ac9a8b987db</guid>
            <category><![CDATA[sre]]></category>
            <dc:creator><![CDATA[Squadcast]]></dc:creator>
            <pubDate>Fri, 01 Mar 2024 06:56:59 GMT</pubDate>
            <atom:updated>2024-03-01T06:56:59.655Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*rzIuq-8QeDeVvwzw.png" /></figure><p>Hiteshwar is an SRE based out of Mumbai, India. His area of specialization is in distributed systems. He works on Kubernetes, running his own custom clusters, maintaining them and creating tools to manage and monitor them. He likes to share his learnings by writing articles and blogs on <a href="https://medium.com/@ihiteshwar">Medium</a> and <a href="https://www.linkedin.com/in/ihiteshwar/">Linkedin</a>. He is an active speaker in meetups and developer groups and also teaches DevOps and SRE practices at learning centers.</p><p><strong>1. How did you become an SRE?</strong></p><p>I loved computers from when I was young. In 10th grade, I started building PCs and troubleshooting tasks by installing a lot of software and games on both Windows and Linux which led me to like Systems Administration. I discovered my love for troubleshooting systems and maintaining them in college when I managed all of their infrastructures.</p><p>As I started my professional journey I was tasked with a lot of automation projects relevant to my role in DevOps. All this experience propelled me into the path of being an SRE.</p><p><strong>2. What’s the most challenging part of your job?</strong></p><p>For me, it’s expecting the unexpected. Quoting John Wilkes of Google “Computer components are very reliable but once you have a lot of them they fail all the time”. To maintain that illusion of stability for external users even when you have 1000s of failure domains internally is probably the hardest thing for an SRE to achieve. So when there is a failure, how fast you detect it, building an automated recovery and finding a fix that is resilient is what keeps me busy.</p><p><strong>3. What process, tools and techniques you can’t live without?</strong></p><p>Currently, I rely on ATOM as my IDE and Ansible for pushing configs around. I am always looking for good articles and open source projects on the web and try to run them in my own dev environments and if possible I try contributing to them as well. I have been using Kubernetes a lot lately both at work and in my personal projects, hence it’s what I am currently most focused on.</p><p><strong>4. What according to you is the future of SRE?</strong></p><p>SREs are here to stay, any organization that cares about its users and systems wants to build a culture around SRE. Finding and hiring a good SRE is hard because apart from being good technically, one has to respect the process and has to have a sense of ownership and passion towards the systems they manage. In my opinion, SRE’s job is a never-ending one because as your business scales, your users scale, your systems scale and so does your failure domains and to keep those running reliably you need someone with incredible passion and grit who is determined to learn new skills and is curious about everything that happens in the tech world.</p><p><strong>5. Any productivity hacks that you would give to new SREs?</strong></p><ul><li>Keep notes of what you learn or if you come across something interesting or challenging along the way.</li><li>Build snippets of code and tools that help you to do your tasks efficiently.</li><li>Always keep revising your fundamentals of OS, Network, Orchestration, Cloud, Monitoring, etc.</li><li>Keep an eye on new and emerging technologies by reading blogs, attending keynotes and deep-dives of various conferences such as SREcon, KubeCon, other CNCF events, VelocityConf etc.</li><li>Try to replicate systems in your dev environments and then try to break them, this will give you an idea of what fails when and how to fix it which is what SRE is all about.<strong>‍</strong></li></ul><p><strong>6. What are some of the things people get wrong about this role?</strong></p><p>SRE has become a buzzword in the tech world. Everyone wants to be an SRE assuming it’s just like system administration and companies are enabling this by renaming SysAdmin jobs to SRE. But the reality is an SRE is a software engineer who knows his way around the Infrastructure that is running the software. SRE understands how that software will behave when the infrastructure running it will fail and how to bring it up if there is a failure.</p><p>With this knowledge, SREs can infuse reliability in software at the design level. So a software designed with reliability and failure domains in mind will be more reliable when taking hits from all sorts of expected and unexpected failures both at the service and infrastructure level.</p><p>Follow the journey of more such inspiring SREs from around the globe through our <a href="https://www.squadcast.com/blog-categories">SRE Speak Series</a>.</p><p><em>Originally published at </em><a href="https://www.squadcast.com/blog/hiteshwar-shares-his-thoughts-on-being-an-sre"><em>https://www.squadcast.com</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9ac9a8b987db" width="1" height="1" alt=""><hr><p><a href="https://medium.com/squadcast/hiteshwar-shares-his-thoughts-on-being-an-sre-9ac9a8b987db">Hiteshwar shares his thoughts on being an SRE</a> was originally published in <a href="https://medium.com/squadcast">Squadcast</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Unlocking Visibility and Control: Introducing Squadcast’s Service Graph Feature]]></title>
            <link>https://medium.com/squadcast/unlocking-visibility-and-control-introducing-squadcasts-service-graph-feature-dedf57dfe11a?source=rss----19ae28e78429---4</link>
            <guid isPermaLink="false">https://medium.com/p/dedf57dfe11a</guid>
            <category><![CDATA[squadcast]]></category>
            <category><![CDATA[product-updates]]></category>
            <dc:creator><![CDATA[Squadcast]]></dc:creator>
            <pubDate>Fri, 23 Feb 2024 11:01:19 GMT</pubDate>
            <atom:updated>2024-02-23T11:01:19.841Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/855/0*9Vi7Mfdd9wHj5aWl.png" /></figure><p>To ensure efficient Incident Management, it is crucial to proactively anticipate and address potential disruptions The need for a comprehensive, high-level view of the status of all services is paramount. Enter Squadcast’s <a href="https://support.squadcast.com/services/service-graph">Service Graph</a> — a feature designed to transform the way organizations approach Incident Management. In this blog, we’ll delve into the intricacies of Squadcast’s Service Graph, exploring its functionalities, benefits, and the game-changing impact it can have on your incident response strategy.</p><h3>Understanding Service Graph: A Visual Representation of Service Dependencies</h3><p>At its core, Service Graph is a mapping tool that provides a visual representation of all your services and their interdependencies. Imagine having a single, connected graph that encapsulates the health status of your services, offering insights into their current state. This visual representation of service dependencies is crucial in understanding the complexity of your infrastructure.</p><h3>Addressing a Common Challenge in Incident Management: The Need for a Holistic View</h3><p>Incident Management teams often grapple with the challenge of obtaining a high-level, at-a-glance view of the status of all services. Service Graph serves as the first node in your incident response process, functioning as a comprehensive service monitoring dashboard. It empowers you to gain a clear overview of the status and impact of everything happening with your services, whether you’re dealing with incidents or striving for optimal service performance.</p><h4>Core Functionalities: Navigating the Service Landscape with Ease</h4><p><strong>View the Upstream and Downstream Relationships:</strong> Service Graph allows you to explore the relationships between your services in one graph. You can understand how different services depend on each other, providing insights into your service ecosystem.</p><p><strong>More Stats on Open Incidents:</strong> Service Graph goes beyond dependency mapping by offering detailed statistics on open incidents associated with your services. This data allows you to pinpoint specific issues and prioritize your response accordingly.</p><p><strong>Please Note: </strong>Service Graph Is not an Analytics Panel andis not intended for in-depth data analysis. Instead, it is designed as a relationship diagram with a health status indication, providing you with a quick and visual overview of your services.</p><h4>Key Benefits: Why Choose Squadcast’s Service Graph?</h4><p><strong>Simplifying Visualizing Complexity:</strong> Squadast’s Service Graph presents a visual representation of intricate systems, simplifying the comprehension of your infrastructure’s architecture.</p><p><strong>Uncovering Root Causes:</strong> When incidents arise, the Service Graph empowers you to trace dependencies back to their origins, aiding in the identification of root causes. This capability is pivotal for quicker resolution and help you better understand the impact radius of an incident</p><p><strong>Scalability Insights:</strong> As your organization expands, Squadast’s Service Graph is an invaluable asset for managing the scalability of your services. It facilitates comprehension of their interactions and load distribution.</p><h3>How to view Service Graphs in Squadcast</h3><p>To view the service graph, Navigate to <strong>Services</strong> and click on <strong>Graph</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*1zseZh4xvKBJDpMp.png" /><figcaption><em>Access Service Graph</em></figcaption></figure><h3>Actions for Service Graph</h3><p>The Service Graph is automatically generated by Squadcast, reflecting the service dependencies you establish.</p><ul><li>Users have the flexibility to zoom in and out, fitting the graph to the screen for optimal viewing.</li><li>Interactivity is enhanced as users can drag graph components for improved visualization and can choose to lock this customized view.</li><li>Additionally, users can opt to display the right panel for the services showcased.</li></ul><h3>Conclusion</h3><p>The Service Graph feature is not merely a tool; it’s a solution to a common challenge in the world of incident management. As organizations rely more heavily on digital services, the need for streamlined incident response becomes increasingly critical. With the Service Graph, you have the power to navigate your service landscape with clarity, respond to incidents with precision, and maintain the reliability of your systems. Try out Service Graph today and experience the benefits of having a clear, connected view of your service ecosystem.‍</p><p>‍<a href="https://www.squadcast.com/"><em>Squadcast</em></a>‍<em> is an Incident Management tool that’s purpose-built for SRE. Get rid of unwanted alerts, receive relevant notifications and integrate with popular ChatOps tools. Work in collaboration using virtual incident war rooms and use automation to eliminate toil.</em></p><figure><a href="https://register.squadcast.com/"><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*MJ1yvVs_Ftlz1f8c.png" /></a></figure><p><em>Originally published at </em><a href="https://www.squadcast.com/blog/unlocking-visibility-and-control-introducing-squadcasts-service-graph-feature"><em>https://www.squadcast.com</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dedf57dfe11a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/squadcast/unlocking-visibility-and-control-introducing-squadcasts-service-graph-feature-dedf57dfe11a">Unlocking Visibility and Control: Introducing Squadcast’s Service Graph Feature</a> was originally published in <a href="https://medium.com/squadcast">Squadcast</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Essentials for the Modern Incident Response Platform]]></title>
            <link>https://medium.com/squadcast/essentials-for-the-modern-incident-response-platform-819cc41f8ad2?source=rss----19ae28e78429---4</link>
            <guid isPermaLink="false">https://medium.com/p/819cc41f8ad2</guid>
            <category><![CDATA[incident-response]]></category>
            <category><![CDATA[best-practices]]></category>
            <dc:creator><![CDATA[Squadcast]]></dc:creator>
            <pubDate>Thu, 15 Feb 2024 05:46:08 GMT</pubDate>
            <atom:updated>2024-02-15T05:46:08.271Z</atom:updated>
            <content:encoded><![CDATA[<p>This article provides site reliability engineers (SREs) with valuable insights into the transformative power of modern incident response platforms. By delving into the must-have features of modern platforms-such as cloud service integrations, service catalogs, single-pane-of-glass visibility, SLO management, and the automation of routine tasks-SREs will learn how these platforms can streamline their incident management processes, enhance operational efficiency, and thus fortify the digital infrastructures of their organizations. The knowledge acquired from this article will empower SREs to make informed decisions when selecting incident management tools and implementing related processes.</p><h3>Summary of key modern incident response platform concepts</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Q8TPkeKAkEqoR1fIV0I-Yg.png" /></figure><h3>Integrations with cloud services</h3><p>In today’s fast-paced digital landscape, incident management tools must be equipped with the latest technologies to effectively handle incidents. One way to enhance these tools is through integration with cloud services. By integrating incident management tools with cloud services, organizations can automate tasks and streamline their incident response workflows. There are numerous benefits of integrating incident management tools with cloud services.</p><p>First, this allows for the convergence of information from multiple sources onto a single screen. This means that SREs can access all relevant data and metrics in one place, making it easier to analyze and respond to incidents efficiently.</p><p>Furthermore, integrating incident management tools with cloud services enables the automation of routine tasks. This not only saves time but also reduces the risk of human error. SREs can focus on more critical aspects of incident response while repetitive tasks are handled automatically.</p><p>Overall, integrating incident management tools with cloud services amplifies their capabilities and improves the efficiency of incident response workflows. It empowers SREs to effectively manage incidents by providing them with a centralized platform that consolidates information and automates routine tasks.</p><h3>Cloud integration benefits</h3><p>Cloud integration provides numerous benefits:</p><ul><li><strong>Centralization of information: </strong>Data from various sources can be converged onto a single platform. SREs no longer need to juggle multiple platforms or sift through disparate data sources because everything they require is right at their fingertips in one unified dashboard.</li><li><strong>Enhanced automation: </strong>Beyond the basic automation of tasks, cloud integration enables advanced automation capabilities. For instance, machine-learning algorithms can be employed to predict potential incidents based on historical data, allowing SREs to address issues even before they arise.</li><li><strong>Scalability: </strong>Cloud services are inherently scalable. As an organization grows, its incident management tools can scale accordingly without the need for significant overhauls or migrations.</li><li><strong>Cost efficiency: </strong>Leveraging cloud integrations can lead to cost savings. By utilizing cloud resources, organizations can avoid the capital expenditures involved in setting up and maintaining physical infrastructure.</li></ul><h3>Single pane of glass</h3><p>In incident management, having a single pane of glass refers to the practice of consolidating all incident information onto one screen or dashboard. This approach offers several advantages for SREs and incident response teams.</p><h3>Improved visibility and accessibility of incident information</h3><p>By bringing together data from various sources onto a single screen, the single pane of glass provides SREs with improved visibility into ongoing incidents. They can quickly assess the severity, impact, and status of each incident without having to navigate through multiple tools or interfaces. This enhanced visibility allows for better prioritization and allocation of resources.</p><p>The single pane of glass also improves accessibility to incident information. SREs can easily access relevant details such as alerts, logs, metrics, and documentation in one centralized location. This eliminates the need to switch between different systems or applications, saving time and effort during incident resolution.</p><h3>Efficient decision-making and collaboration</h3><p>With all incident information available on one screen, SREs can make faster and more informed decisions. They can analyze data holistically, identify patterns or trends across incidents, and take appropriate actions accordingly. The ability to view real-time updates on ongoing incidents enables quick response times and minimizes downtime.</p><p>Additionally, the single pane of glass promotes efficient collaboration among team members. By having all relevant information in one place, SREs can easily share insights, communicate progress updates, and coordinate their efforts effectively. This streamlined collaboration enhances teamwork and ensures that everyone is aligned toward resolving incidents promptly.</p><h3>Reduced incident resolution time and effort</h3><p>The consolidated view provided by the single pane of glass reduces the time and effort required for incident resolution. SREs no longer need to search for information across multiple tools or platforms; everything they need is readily available on one screen. This simplifies the troubleshooting process, accelerates root cause analysis, and facilitates faster incident resolution.</p><p>Furthermore, the single pane of glass eliminates the need for manual data aggregation and correlation. By automatically integrating information from various sources, it reduces the risk of human error and ensures that SREs have a comprehensive understanding of each incident. This efficiency translates into reduced downtime and improved service reliability.</p><figure><a href="https://register.squadcast.com/"><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2XbK5iC1-cY42Gp-sl_u6Q.png" /></a></figure><h3>Features and innovations of modern incident management tools</h3><p>Modern incident management tools have evolved significantly from their traditional counterparts, offering a range of features and innovations that cater to the dynamic needs of today’s digital landscape. Let’s explore some of the key features that set these tools apart and make them an essential asset for site reliability engineers and incident response teams.</p><h3>Integration capabilities</h3><p>Modern incident management tools provide seamless integration with various cloud services, enabling organizations to connect their existing systems and tools, thereby facilitating smooth data flow and the automation of incident management workflows.</p><h3>ChatOps tools</h3><p>These tools integrate with popular collaboration platforms such as Slack, Microsoft Teams, and Google Hangouts, allowing SREs to route incident updates to dedicated ChatOps channels or custom communication channels. This ensures effective communication during incident response and facilitates transparent discussions among team members.</p><h3>Retrospectives</h3><p>Retrospectives are a valuable tool for continually enhancing incident response processes and infrastructure. By conducting retrospectives effectively, you can mitigate risks, improve accountability, and foster a culture of learning, leading to happier team members.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*DB0kmv13auAD3mz7.png" /><figcaption>Squadcast’s Retrospective Templates</figcaption></figure><h3>Service catalogs</h3><p>A <a href="https://www.squadcast.com/service-catalog">service catalog</a> offers a unified view of all active services in an intuitive dashboard, complete with service health and dependencies. This feature promotes responsible engineering by enabling service ownership and visibility, restricting the impact on affected services. It also serves as a single source of truth for service health, allowing you to effectively monitor real-time metrics and incident data.</p><h3>Slack integration</h3><p>Some tools offer Slack integration, providing valuable insights into on-call schedules and allowing incidents to be triggered directly from within Slack. This streamlines the incident creation process and fosters better teamwork through collaborative, incident-specific Slack channels.</p><h3>Bidirectional sync</h3><p>Bidirectional sync features enable users to perform response actions without leaving the platform, such as creating tickets automatically in Jira, rebuilding projects, or performing rollbacks via build/deployment tools. Runbooks can be used to document response actions for routine tasks, and custom APIs and webhooks provide flexibility in executing scripts or integrating with other systems as needed.</p><h3>Role-based access control (RBAC)</h3><p>Role-based access control (RBAC) is a crucial aspect of any incident management platform, ensuring that individuals have access only to the data and resources necessary for their specific roles. This is vital for safeguarding sensitive information while facilitating efficient incident resolution.</p><p>To fulfill access needs while ensuring data security, Squadcast offers <a href="https://support.squadcast.com/manage-teams/role-based-access-control">role-based access control (RBAC)</a>. Users can be grouped into specific teams, and roles can be assigned to each member. Customizable user permissions at the organization and team levels provide granular control over access privileges.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*phwQLOOG0-zRJq9c.png" /><figcaption>Squadcast’s role-based access control (RBAC)</figcaption></figure><p>Squadcast sets itself apart from traditional incident management tools through its integration capabilities, ChatOps tools, slack bot integration, bidirectional sync feature, and RBAC functionality. These features and innovations empower SREs to streamline their incident response processes and collaborate effectively for faster incident resolution.</p><h3>Status pages</h3><p><a href="https://www.squadcast.com/statuspage">Status pages</a> ensure that SREs remain informed about unplanned downtimes and outages regardless of where you manage your incident response processes. A status page would ideally be fully customizable and serve as the ultimate source of truth for your system’s service status. By leveraging status pages, you build trust with your customers and can also achieve significant cost savings while delivering transparency and reliability.</p><h3>Automation and routine task elimination</h3><p>Automation plays a crucial role in incident management by streamlining workflows and improving the efficiency of incident response. Automating repetitive and time-consuming tasks provides organizations with faster incident resolution, less human error, and better overall efficiency.</p><p>The various benefits of streamlining incident management workflows are described below.</p><h3>Automating repetitive and time-consuming tasks</h3><p>Incident management often involves several routine tasks that can be automated to save time and effort. For example, automatically creating tickets in ticketing systems like Jira or ServiceNow when an incident is triggered eliminates the need for manual ticket creation. Similarly, automating the gathering of relevant logs, metrics, and other diagnostic information can significantly speed up the troubleshooting process.</p><p>SREs can leverage automation to focus on critical aspects of incident response instead of getting bogged down by repetitive tasks. This improves their productivity and ensures consistent and accurate execution of these tasks.</p><h3>Enabling faster incident response and resolution</h3><p>Automation enables faster incident response by reducing manual intervention and accelerating the overall incident resolution process. When incidents occur, automated notifications can be sent to the appropriate stakeholders or on-call responders instantly. This ensures that incidents are addressed promptly without delays caused by manual communication processes.</p><p>Automation allows for quicker data analysis and correlation as well. By automatically aggregating data from various sources, such as monitoring tools or log management systems, SREs can gain insights into the root cause of incidents more rapidly, so they can take appropriate actions swiftly and minimize downtime.</p><p>To explore Squadcast’s automation capabilities and discover how they enhance incident response, visit <a href="https://www.squadcast.com/statuspage">Squadcast’s Status Page</a>.</p><h3>Reducing human error and improving overall efficiency</h3><p>Automating routine tasks reduces the risk of human error during incident management. Manual processes are prone to mistakes due to factors like fatigue or oversight. Automating these tasks lets organizations ensure consistency in execution while minimizing errors caused by human factors.</p><p>Moreover, automation delivers a substantial boost to overall efficiency by eliminating bottlenecks within the incident management workflow. It slashes the time and effort required for repetitive tasks, enabling SREs to concentrate on critical aspects of incident response. This translates into faster incident resolution, improved service reliability, and, ultimately, heightened customer satisfaction.</p><p>For a comprehensive solution that complements automation and optimizes your service-level objective (SLO) tracking and error budget management, consider Squadcast’s cutting-edge offerings. Squadcast’s SLO Tracker lets you define your target SLO and leverage corresponding service-level indicators (SLIs) to effortlessly track error budget burn rates without the complexity of configuring and aggregating multiple data sources. Discover how Squadcast can enhance your incident management process at <a href="https://www.squadcast.com/slo-tracker">Squadcast’s SLO Tracker.</a></p><h3>How to maximize an incident management tool’s utility</h3><p>First and foremost, it is crucial to select a tool that seamlessly integrates with your existing systems. This ensures consistent and effective data flow, reducing the need for manual intervention.</p><p>Scalability is another important factor to consider. Your incident response tool should be able to handle increased data, users, and incidents efficiently without performance degradation. Additionally, the tool should have the ability to guarantee service-level objectives and rate limits for monitoring mission-critical applications.</p><p>Effective alert management is essential in incident response. Look for a tool that helps you sift through the noise and prioritize critical alerts. Features like alert aggregation, deduplication, suppression, prioritization, and routing rules can significantly aid in managing high volumes of alerts.</p><p>Real-time collaboration is vital for quick and effective incident response. Choose a tool that fosters real-time collaboration among the various teams involved in incident management. Integrated chat, conference bridges, and collaborative dashboards can facilitate seamless communication during incidents.</p><p>By considering these factors when selecting an incident management tool, you can maximize its utility and ensure efficient incident response processes within your organization.</p><figure><a href="https://register.squadcast.com/"><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yvrWjVk9D7QtKzzhpHGyeg.png" /></a></figure><h3>Conclusion</h3><p>Modern incident response platforms, like <a href="https://www.squadcast.com/">Squadcast</a>, are revolutionizing how organizations handle and manage incidents. By integrating with cloud services, offering a single pane of glass for consolidated information, and automating routine tasks, these platforms are streamlining the incident management workflow, making it more efficient and effective.</p><p>Site reliability engineers looking to stay ahead in this fast-paced digital landscape must understand the capabilities and features of these modern tools. Squadcast, with its innovative features and integrations, offers a comprehensive solution that sets it apart from traditional incident management tools.</p><p>By leveraging the power of these platforms and implementing strategies to maximize their utility, SREs can ensure faster incident resolution, reduced downtime, and improved service reliability. As the digital landscape continues to evolve, having a robust incident response platform will be crucial for organizations aiming to deliver seamless digital experiences to their users.</p><p>The key to effective incident management lies in the combination of advanced tools, streamlined workflows, and skilled professionals working in harmony. With the right tools in hand and a clear understanding of their capabilities, SREs can confidently navigate the challenges of incident management and ensure the smooth operation of their digital services.</p><p><em>Originally published at </em><a href="https://www.squadcast.com/incident-response-tools/modern-incident-response-platform"><em>https://www.squadcast.com</em></a><em> on November 17, 2023.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=819cc41f8ad2" width="1" height="1" alt=""><hr><p><a href="https://medium.com/squadcast/essentials-for-the-modern-incident-response-platform-819cc41f8ad2">Essentials for the Modern Incident Response Platform</a> was originally published in <a href="https://medium.com/squadcast">Squadcast</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Suppressing Alert Noise during Scheduled Maintenance]]></title>
            <link>https://medium.com/squadcast/suppressing-alert-noise-during-scheduled-maintenance-2b0581bd6651?source=rss----19ae28e78429---4</link>
            <guid isPermaLink="false">https://medium.com/p/2b0581bd6651</guid>
            <category><![CDATA[sre]]></category>
            <category><![CDATA[on-call]]></category>
            <category><![CDATA[best-practices]]></category>
            <dc:creator><![CDATA[Squadcast]]></dc:creator>
            <pubDate>Fri, 02 Feb 2024 06:05:35 GMT</pubDate>
            <atom:updated>2024-02-02T06:05:34.994Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/855/0*LLWyWkFN2wSqO72p.png" /></figure><p>Alert noise is a common problem for IT teams that monitor and manage complex systems. Excessive unactionable alerts triggered by various sources, such as applications, servers, network devices, etc., can cause alert fatigue. The higher volume of alerts can be overwhelming, reducing the ability to respond to critical alerts.</p><p>One <strong>event of possible alert noise is during scheduled maintenance, a</strong>which is a common practice in the digital realm<strong>.</strong></p><h3>Problems at Hand</h3><p>Maintaining operational continuity is non-negotiable. Here are a few common use cases that demonstrate the challenges of alert noise faced during scheduled maintenance.</p><ul><li>Proactively muting all alerts from specific sources like Datadog.</li><li>Muting Specific alerts from monitoring tools like Prometheus, Datadog, or New Relic while enhancing API sets.</li><li>Suppress a high volume of alerts while conducting load testing on a web application or server.</li><li>Known anomalies or irregularities in certain systems or processes that generate alerts.</li></ul><h3>Suppression Rules</h3><p>Squadcast’s suppression rules offer granular control for muting alerts from various sources. You can use these rules to focus on maintenance tasks without getting overwhelmed by unactionable alerts. They can be applied to entire services, multiple alert sources, specific variables, or combinations to suit your needs.</p><h3>How To Configure Suppression Rules?</h3><p>You can suppress unwanted alerts with <strong>Alert Suppression</strong>.</p><ul><li><strong>Every service in Squadcast supports Alert Suppression</strong> allowing you to create suppression rules by choosing an alert source or a host and add conditions to suppress alerts by <strong>time</strong>.</li><li>The ‘time’ in this case will be the maintenance window<strong> you’ve anticipated for your maintenance.</strong></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*o-iq6Hjm2hD1a2sp.png" /></figure><ul><li>Within the service, if you <strong>receive notifications from a selected alert source for the set period,</strong> the alert notifications shall be suppressed.</li><li>If you are aware that the payload contains variables specific to the API currently being worked on, you have the option to employ a suppression rule that targets that particular API, allowing you to temporarily suppress alerts of that nature.</li><li>Similarly you can <strong>add new conditions and rules.</strong> This is an efficient way to control alert noise when you’re running maintenance at your end.</li><li><strong>Host-specific suppression</strong> ensures that your monitoring process for the rest of the system remains intact and unhindered.</li></ul><p><strong>Impact:</strong> Alert Suppression reduces alert noise during maintenance ensuring you receive only important alerts. However, it’s important to note that</p><ul><li>Suppressed incidents<strong> cannot be</strong> acknowledged, reassigned, or resolved.</li><li>Post mortems are not available for suppressed incidents.</li><li>Further customization and configuration of your suppression rules can be done using REST APIs.</li></ul><p>For more information, explore the short video on <a href="https://www.youtube.com/watch?v=Uzu6Yk-VjSY">suppression rules</a> &amp; configuration options for <a href="https://support.squadcast.com/services/alert-suppression">alert suppression</a> and <a href="https://support.squadcast.com/services/maintenance-mode">maintenance mode</a> in Squadcast.</p><h3>Conclusion</h3><p>Any downtime or disruption can have a ripple effect on your business. But what happens when you need to <strong>make improvements or perform maintenance on a specific part of your infrastructure without causing unnecessary alert noise</strong>?</p><p>Squadcast offers a <strong>practical solution through automation rules, specifically suppression rules. </strong>These rules allow you to gain <strong>granular control </strong>over alert noise during maintenance periods <strong>without sacrificing overall system monitoring.</strong></p><p>So, you can now create an environment that<strong> promotes focused Incident Response and facilitates an enhanced Incident Management experience</strong>.</p><p><a href="https://www.squadcast.com/"><em>Squadcast</em></a><em> is a Reliability Workflow platform that integrates On-Call alerting and Incident Management along with SRE workflows in one offering. Designed for a zero-friction setup, ease of use and clean UI, it helps developers, SREs and On-Call teams proactively respond to outages and create a culture of learning and continuous improvement.</em></p><figure><a href="https://register.squadcast.com/"><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*uEIonFyT7vRl_UPC.png" /></a></figure><p>‍</p><p><em>Originally published at </em><a href="https://www.squadcast.com/blog/suppressing-alert-noise-during-scheduled-maintenance"><em>https://www.squadcast.com</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2b0581bd6651" width="1" height="1" alt=""><hr><p><a href="https://medium.com/squadcast/suppressing-alert-noise-during-scheduled-maintenance-2b0581bd6651">Suppressing Alert Noise during Scheduled Maintenance</a> was originally published in <a href="https://medium.com/squadcast">Squadcast</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Leading Incident Management Best Practices]]></title>
            <link>https://medium.com/squadcast/leading-incident-management-best-practices-e5623e130844?source=rss----19ae28e78429---4</link>
            <guid isPermaLink="false">https://medium.com/p/e5623e130844</guid>
            <category><![CDATA[incident-management]]></category>
            <category><![CDATA[best-practices]]></category>
            <dc:creator><![CDATA[Squadcast]]></dc:creator>
            <pubDate>Thu, 25 Jan 2024 07:08:30 GMT</pubDate>
            <atom:updated>2024-01-25T07:08:30.294Z</atom:updated>
            <content:encoded><![CDATA[<p>Being prepared for unexpected events is crucial. Incidents can significantly disrupt businesses and cause harm to stakeholders and customers alike, so preventing them, whether they are technical or caused by human error, is essential for maintaining business continuity and retaining customer trust. And when an incident does occur-such as a failed backup job, a power outage, or even a full-blown ransomware attack-handling and managing it properly is pivotal to the survival and prosperity of your organization.</p><p>A site reliability engineer (SRE) working for any corporate entity, be it a managed services provider (MSP), a small or medium-sized business, or a large enterprise, should define what constitutes an incident and proactively put forth efforts toward prevention and handling. Then, as engineers, they must prepare for the inevitable and have the right tools and information at hand to resolve whatever comes up.</p><p>This article dives into the top ten best Incident Management Practices, showcasing the lifecycle of an incident and a roadmap for successful incident resolution.</p><h3>Key Incident Management best practices</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KyeZ5GYxpeUik0__fpNNfw.png" /></figure><h3>The lifecycle of an incident</h3><p>As an SRE, it’s critical to have a complete understanding of the lifecycle of an incident, which encompasses 10 steps:</p><ol><li><strong>Detection:</strong> An issue is detected and logged using a tool of your choice designed for this task.</li><li><strong>Reporting:</strong> The incident is reported and communicated to the designated personnel.</li><li><strong>Response:</strong> Engineers work to resolve the incident.</li><li><strong>Communication:</strong> The team communicates regular updates to stakeholders via agreed communication channels.</li><li><strong>Resolution:</strong> The issue is resolved by configuration changes or other remediation.</li><li><strong>Post-incident review:</strong> Action items are documented, and a root cause analysis (RCA) is conducted.</li><li><strong>Documentation and knowledge sharing:</strong> Lessons learned are recorded and shared with the team.</li><li><strong>Monitoring and followup:</strong> The team ensures the system’s stability and implements prevention measures.</li><li><strong>Closure:</strong> The incident is considered closed.</li><li><strong>Post-mortem:</strong> A written record of the incident, its impact, actions taken by the people involved, and follow up actions are collectively documented and discussed among team members.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*qPCWX2pXpLoiqdAy.png" /><figcaption>The 10 stages of the incident lifecycle</figcaption></figure><h3>Top ten incident management best practices in detail</h3><p>There are many different opinions on the best practices for handling incidents. Organizations may follow different procedures, and different products dictate specific approaches. That said, there are common themes that apply widely. Let’s dive into the top ten best practices in more detail to understand them clearly.</p><h3>Have the right people assigned to the incident</h3><p>A decision-maker must make an effort to form an incident response task force consisting of the right set of knowledgeable, experienced, and responsible people. This group should be multidisciplinary, experienced, and responsible and should have a proven track record.</p><p>Access and privilege rights must already be in place to avoid delays and increased response times. The member roles of such a team should include an infrastructure specialist, the application owner, a subject matter expert (SME) relevant to the technologies used, and an SRE with access to monitoring, alerting, and communication tools.</p><p>Ideally, the members of the team should know each other and be able to communicate directly, precisely, and in a timely manner. There must be some overlap in their skills to augment each other, but they must have distinct areas of responsibility for resolving the issue.</p><figure><a href="https://register.squadcast.com/"><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Pny6CErUiLhG2GQ5kEvtoQ.png" /></a></figure><h3>Establish proper communication channels</h3><p>Coordination channels must be in place to allow individuals within their respective teams to coordinate their efforts and exchange information.</p><p>During the lifecycle of an incident, everyone needs to be on the same page. Confusion can be detrimental, resulting in data losses or adding costs for the organization.</p><p>One or more predefined lists of people should be compiled, and the correct information must be disseminated to the right list, increasing incident status visibility and reducing noise during incident handling.</p><h3>Use tools to detect and report incidents</h3><p>It is necessary to use specialized tools that allow you to set and aggregate alerts, define thresholds, and integrate with other tools in your arsenal.</p><p>Ensure that your tools provide multiple notification methods, including SMS, push notifications, emails, or phone calls. Create dashboards and status pages to give overviews of incident status and ensure that you are aware of scheduled events that may have a maintenance window or other planned downtime.</p><p>In addition, the tools of your choice should provide an easy way for you to know what jobs are scheduled during the timeframe of your incident in order to reschedule or mitigate resources accordingly.</p><h3>Define what is considered an incident for your organization</h3><p>When you have an event or occurrence that impacts the reliability of an application, service, or system by affecting its availability or performance, you might have an incident. Labeling a non-incident as an incident during reduced reliability is easy, but not everything falls under the incident category. Using your monitoring tools, you will notice events that warn you of imminent incidents, downtime, and loss of connectivity.</p><p>In any organization, team leads should periodically run a capacity planning exercise to ensure that the system can handle unexpectedly high loads. SREs must pay attention to the monitoring tools and the alerts and warnings they generate while being aware of scheduled maintenance windows and thoroughly understanding what constitutes a problem as opposed to an incident.</p><p>While the exact choice will depend on the organization, the table below shows some typical differentiations.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-h4q5lGf4Gs4K0s7ZOiAYg.png" /></figure><h3>Identify an incident manager</h3><p>An incident manager acts as a centralized communication, coordination, and decision-making hub. This person will prioritize tasks, approve or deny actions, and be accountable for the outcome.</p><p>The incident manager is often the one keeping detailed records for post-incident analysis. An incident manager also reviews the documentation produced during and after the incident to ensure accuracy.</p><h3>Build a solid knowledge base and extend it as required with each incident</h3><p>Having, updating, and maintaining an easy-to-navigate knowledge base is most important because having one that can be read and reviewed by task force members has been proven to reduce incident resolution times. It also aids in knowledge sharing, thus improving the team’s efficiency.</p><p>Remember that the knowledge base must be structured to be easy to search and navigate, no matter which application you use to create it.</p><h3>Know your SLOs and keep an eye on your SLAs</h3><p>Knowing and tracking your service-level objectives (SLOs) keeps you focused on what you need to deliver regarding performance, reliability, and other business-set criteria. Incidents happen, but as long as SLOs are met and service-level agreements (SLAs) are kept, it’s all business as usual.</p><h3>Automate everything possible and have runbooks where human intervention is needed</h3><p>When it comes to managing an incident, automation is critical. SREs and incident managers should always be working on identifying and implementing automation opportunities. These can cover a broad spectrum of services and functions, some technical and others procedural.</p><p>Automation opportunities commonly arise in these areas:</p><ul><li>Alerting</li><li>Prioritization</li><li>Notification</li><li>Escalation</li><li>Mitigation</li><li>Scaling and allocation of resources</li><li>Integrations with security information and event management (SIEM) platforms</li></ul><h3>Note actions taken and conclusions made as they happen</h3><p>Collecting information while the incident is happening and creating documentation as the lifecycle progresses make life easier when the time comes for the post-mortem.</p><p>A post-mortem is one of the most important ceremonies of an incident. Still, with the incident behind you, it’s easy to forget details of what happened during the incident resolution. It’s always beneficial to document your actions and note conclusions, potential automation opportunities, and optimizations in current runbooks while working to make the subsequent incident resolution faster and painless.</p><h3>Keep actions blameless</h3><p>A blameless culture reduces anxiety in teams and individuals, improves collaboration, and retains talent. In a transparent environment, people tend to be more accountable.</p><p>Trial and error is encouraged, leading to increased innovation. In an environment where people are not worrying about being blamed, trust is built along with positive relationships and work enjoyment, contributing to successful and quick incident resolution.</p><figure><a href="https://register.squadcast.com/"><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AkTcbfgX0Y8ozrT8-Ymdsw.png" /></a></figure><h3>Conclusion</h3><p>Following the correct practices and utilizing the right tools is the recipe for successful incident resolution. The above-mentioned techniques, procedures, and methodologies will reduce incidents and streamline incident management as time passes.</p><p>Having a centralized platform where task force management, incident response, coordination, alerting, and monitoring work together harmoniously is bound to reduce toil and keep error budgets in check. Effective communication channels and native integrations with the most popular tools in the market will guarantee a positive, enhanced experience during an incident.</p><p>Squadcast integrates with almost 100 monitoring tools, aggregates information for reliable alerting, and provides notifications for scheduled maintenance windows. In addition, escalation policies can be set using a combination of rules to alert the right people at the right time via email, push notifications, SMS, or phone calls. In addition, you can have a <a href="https://www.squadcast.com/statuspage">status page</a> where all interested parties can get an overview of where the task force currently is within the incident lifecycle.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*z3Y2PBNdHqxB_IiR.png" /><figcaption>Squadcast’s Status Page</figcaption></figure><p>SREs can also interact with Squadcast’s REST API, use webhooks to send incident information to any platform imaginable, and create or manage resources at scale using the official Terraform provider available at <a href="https://registry.terraform.io/providers/SquadcastHub/squadcast/latest/docs">Terraform’s registry</a> with only a few lines of code.</p><p><em>Originally published at </em><a href="https://www.squadcast.com/incident-response-tools/incident-management-best-practices"><em>https://www.squadcast.com</em></a><em> on November 1, 2023.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e5623e130844" width="1" height="1" alt=""><hr><p><a href="https://medium.com/squadcast/leading-incident-management-best-practices-e5623e130844">Leading Incident Management Best Practices</a> was originally published in <a href="https://medium.com/squadcast">Squadcast</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Unveiling Past Incidents: Accelerating Incident Resolution with Historical Context]]></title>
            <link>https://medium.com/squadcast/unveiling-past-incidents-accelerating-incident-resolution-with-historical-context-e26526950679?source=rss----19ae28e78429---4</link>
            <guid isPermaLink="false">https://medium.com/p/e26526950679</guid>
            <category><![CDATA[squadcast]]></category>
            <category><![CDATA[product-updates]]></category>
            <dc:creator><![CDATA[Squadcast]]></dc:creator>
            <pubDate>Fri, 19 Jan 2024 06:58:26 GMT</pubDate>
            <atom:updated>2024-01-19T06:58:26.388Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/855/0*5GHErybMnbI5p6fT.png" /></figure><p>Having the context of how similar issues were handled in the past can be invaluable. It can help incident responders grasp the nature of recurring problems, their causes, and effective solutions that have worked in the past.</p><p>Introducing <a href="https://support.squadcast.com/incidents-page/past-incidents">Squadcast’s Past Incidents</a> feature that assists incident responders by presenting them with a list of similar past incidents related to the same service they are currently investigating. It employs a system algorithm to match and display similar historical incidents, offering valuable insights such as historical context, past incident activities &amp; involvement, timing patterns, and previously successful solutions aiding faster Incident Resolution for organizations</p><p>This feature is available to everyone who takes Squadcast on a spin via our <a href="http://register.squadcast.com/">14 day free trial</a>!</p><h3>Benefits: Why Past Incidents Matter</h3><p>Here are some reasons that make Past incidents a valuable feature:</p><p><strong>Historical Context and Knowledge Transfer:</strong> Past incidents offer context and a record of how similar issues were handled in the past helping incident responders understand the nature of recurring problems, their causes, and the previously used resolutions. The historical context also allows users to transfer knowledge and insights gained from previous incidents to those who may not have been present during the initial incidents.</p><p><strong>Performance Evaluation:</strong> Users can use past incidents as a benchmark to evaluate their performance in resolving similar issues over time. This helps in identifying trends, areas for improvement, and whether response times and resolutions are improving or deteriorating.</p><p><strong>Documentation:</strong> Past incidents can help teams document the entire Incident Management process, from detection to resolution, making it highly valuable for auditing, compliance, and reporting purposes.</p><p><strong>Incident Analysis:</strong> Analyzing past incidents can help in identifying patterns and trends that may indicate potential future issues. This enables proactive measures to prevent incidents or minimize their impact.</p><p><strong>Decision-Making:</strong> When a new incident occurs, having access to past incident data can aid decision-making. Teams can quickly assess if the current incident is similar to past incidents and use previous incident data to make informed decisions about how to respond.</p><p><strong>Communication:</strong> When communicating with stakeholders, having a history of past incidents can help provide context and explanations for recurring issues, which can be particularly useful in managing stakeholder expectations.</p><h3>View Past Incidents</h3><p>Here is how users can view Past Incidents for a parent incident,</p><p><strong>Step1:</strong> Go to the<strong> Incidents List</strong> page and select the incident you’re interested in from the list.</p><p><strong>Step2:</strong> In the <strong>Incident Details</strong> page, locate the <strong>Details</strong> section.</p><p><strong>Step3:</strong> Look for the <strong>Past Incidents</strong> subsection. And the following information will be displayed</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*lLmQyhMBlaZ78akI.png" /><figcaption><em>Past Incidents on the Incident Details page</em></figcaption></figure><h3>Past Incident Details</h3><p>Here are the details shown for the relevant incidents. By default the top 5 incidents that match and have been the most recent in the last 6 months will be displayed. Responders can now view and access which related incidents had high levels of activity along with the remediation steps taken by responders on these incidents.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*vTjW1YJFAOdK_Xw2.png" /><figcaption><em>Top 5 Relevant Incidents</em></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Y2kUze2oIsuWyLGeSQdsBw.png" /></figure><h3>Incident Statistics</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*7c84YzIxEAFE7TJ3.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GMNL2NV-R5cBTGToO5QPCA.png" /></figure><h3>Conclusion</h3><p>In summary, past incidents serve as a valuable source of information, knowledge, and context for Squadcast users. They play a pivotal role in incident resolution, performance enhancement, and proactive Incident Management, ultimately resulting in more streamlined and efficient incident response procedures.</p><p>At Squadcast, a core tenet of our mission is to ensure that our platform and services offer the utmost user-friendliness, providing you with a seamless experience. We trust that our Past Incidents feature will bring us one step closer to achieving this objective. We encourage you to give this new feature a try and share your thoughts or feedback in the comments or with our support team. Cheers!‍</p><p>‍<em> </em>‍<a href="https://www.squadcast.com/"><em>Squadcast</em></a><em> is a Reliability Workflow platform that integrates On-Call alerting and Incident Management along with SRE workflows in one offering. Designed for a zero-friction setup, ease of use and clean UI, it helps developers, SREs and On-Call teams proactively respond to outages and create a culture of learning and continuous improvement.</em></p><figure><a href="https://register.squadcast.com/"><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Hc7gtp_W2vAUWdSX.png" /></a></figure><p><em>Originally published at </em><a href="https://www.squadcast.com/blog/unveiling-past-incidents-accelerating-incident-resolution-with-historical-context"><em>https://www.squadcast.com</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e26526950679" width="1" height="1" alt=""><hr><p><a href="https://medium.com/squadcast/unveiling-past-incidents-accelerating-incident-resolution-with-historical-context-e26526950679">Unveiling Past Incidents: Accelerating Incident Resolution with Historical Context</a> was originally published in <a href="https://medium.com/squadcast">Squadcast</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Incident Management KPI Best Practices]]></title>
            <link>https://medium.com/squadcast/incident-management-kpi-best-practices-b1ad63e44cc6?source=rss----19ae28e78429---4</link>
            <guid isPermaLink="false">https://medium.com/p/b1ad63e44cc6</guid>
            <category><![CDATA[best-practices]]></category>
            <category><![CDATA[incident-management]]></category>
            <dc:creator><![CDATA[Squadcast]]></dc:creator>
            <pubDate>Wed, 10 Jan 2024 12:17:01 GMT</pubDate>
            <atom:updated>2024-01-10T12:17:01.216Z</atom:updated>
            <content:encoded><![CDATA[<p>As you integrate Site Reliability Engineering (SRE) <a href="https://www.squadcast.com/sre-best-practices">best practices</a> into your organizational framework, monitoring the efficiency of your incident management process becomes crucial. This forward-thinking approach is essential to a mature <a href="https://www.squadcast.com/blog/the-evolution-of-incident-management-from-on-call-to-sre">incident management</a> strategy, and incident management KPIs are the cornerstone of effectively monitoring performance.</p><p>Key Performance Indicators (KPIs) are quantitative metrics that enable the evaluation of your processes, activities, and services’ progression compared to your organization’s strategic objectives. Whether your KPIs are operational or strategic, their real value lies in their ability to provide clear, objective insights into your incident management effectiveness.</p><p>This article will review the importance of utilizing incident management KPIs, how they help measure the effectiveness of your current incident management processes and enable continuous improvement, and best practices to leverage these metrics wisely.</p><h3>Summary of key incident management KPI best practices</h3><p>While recommended practices may differ for different use cases, the following best practices — which we’ll explore later in this article — provide a solid baseline for effectively implementing incident management KPIs in an organization.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YnkunNgDqMXq5P3QBtfdgg.png" /></figure><h3>The role incident management KPIs play</h3><p>Most successful enterprises are known to make strategic decisions based on KPIs that eventually help them pivot from reactive responses into proactive strategies. For instance, imagine the IT team of a large enterprise working through a backlog of incidents. They could tackle them mindlessly or leverage KPIs to identify patterns and achieve an iterative enhancement cycle for Continual Service Improvement (CSI).</p><p>But the effective use of KPIs requires careful consideration of several factors.</p><p>Remember that KPIs are not static. They should evolve with your business. If you notice that a particular KPI is consistently being met with ease, it might be time to revise your targets or introduce a new, more challenging KPI. Similarly, if a KPI is consistently missed, it may indicate that your process or resources need an overhaul.</p><p>The SLA adherence KPI is another significant indicator of your service delivery process. As part of the regular review process, if you find SLA breaches becoming common, it’s essential to identify the root cause. Is the issue with the resource allocation, or are the agreed SLAs unrealistic?</p><p>It’s also crucial to remain disciplined and avoid going overboard with all the potential KPIs you <em>could </em>track. Be selective and choose only those KPIs that best reflect your goals and provide actionable insights.</p><h3>Key advanced incident management KPIs</h3><p>The four advanced incident management KPIs below can help organizations take their incident management practices to the next level.</p><h4>Percentage of Incidents Resolved Remotely (PIRR)</h4><p>Think about the volume of incidents your team handles remotely over the total number of incidents. This rate should make sense in the context of your operations.</p><p>For starters, maintaining an acceptable PIRR means you’re efficiently resolving issues without the need to send technicians to a physical (on-site) location, such as a customer’s office, a data center or any other location where the system or service is deployed. By contrast, incidents resolved remotely are those where the issue can be sorted out without needing to physically be at the location where the problem occurred. This could involve remote desktop control, customer support calls, or resolving server issues from a centralized location.</p><p>A higher PIRR helps quantify how often your team is able to solve problems without the need for time-consuming, costly on-site visits, and is typically a good indicator of efficient operations, as long as quality isn’t compromised.</p><p>But be careful. Extreme spikes or dips in PIRR may indicate overlooked issues that require attention and could be a sign that some issues are not being addressed as they should be.</p><p><strong>Recurring incidents percentage</strong></p><p>Some incidents may consistently return, no matter how often you have resolved them. The recurring incidents percentage metric looks at how often you’re seeing the same issues crop up, highlighting the need for more in-depth investigations.</p><p>A high percentage of recurring incidents commonly suggest that existing resolutions are essentially band-aid solutions and aren’t effectively addressing the underlying systemic problem. A high rate here should also prompt you to investigate and rectify the effectiveness of the incident resolution and prevention mechanisms.</p><h4>Ratio of incidents to problems</h4><p>Are your team’s efforts equally distributed between problem (root cause) analysis as much as resolving incidents? To assess this, consider measuring the number of incidents relative to the number of root causes you’ve identified.</p><p>Unlike the <strong>Recurring Incidents Percentage</strong>, which zeroes in on specific, repeating issues, a high <strong>Ratio of Incidents to Problems</strong> provides a broader view by suggesting that your team spends more time addressing symptoms (incidents) than identifying and resolving the root cause (problems). This approach may not necessarily involve the same recurring issues but suggests that the incident management approach is symptom-focused, instead of being problem-focused. Such an imbalance can potentially make your problem identification and resolution process inefficient, ultimately leading to repeat incidents with varied or even the same root cause.</p><p><strong>Service level objectives (SLOs)</strong></p><p>SLOs offers a pre-defined, nuanced view of service quality and reliability, encompassing various metrics that could affect Customer Satisfaction (CSAT) scores indirectly. <br>For instance, if you notice that your SLO budget has been significantly depleted over the last month, this could be an indicator of multiple factors: it might mean your product has bugs, or that a new feature is causing complications. Alternatively, it could also point that your incident response time needs a re-evaluation, affecting the time it takes to get back to operational normalcy after an issue.</p><p>A high or low SLO can often preemptively signal the need for adjustments in your incident management strategy before it cascades into customer complaints or service level agreement (SLA) violations.</p><figure><a href="https://register.squadcast.com/"><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7EBOyFaOXi3xyhfAcTyXyg.png" /></a></figure><h3>Four essential incident management KPI best practices</h3><p>The four best practices below can help organizations optimize their use of incident management KPIs and improve performance throughout the <a href="https://www.squadcast.com/blog/overview-of-incident-lifecycle-in-sre">incident lifecycle</a>.</p><h3>Incident management KPI best practice #1: Implement data standardization &amp; visualization</h3><p>KPIs are only as good as the data that informs them. Before you start tracking KPIs, ensure that the data you’re pulling from is uniform and accurate.</p><p>For instance, you may track KPIs like <a href="https://www.squadcast.com/blog/how-squadcast-actions-help-you-reduce-mttr">mean time to resolve (MTTR)</a>, first call resolution (FCR) rate, incident recurrence rate, and SLA adherence. Each of these KPIs may be measured on different scales. MTTR might be in hours or days, FCR and SLA Adherence are in percentages, and the incident recurrence rate could be a raw count.</p><p>Analyzing these KPIs without standardizing them onto a common scale can be misleading. Large numbers from one KPI might dwarf smaller but more significant variations in another, leading to skewed interpretations.</p><p>Data normalization helps bring these different KPIs onto a common scale, ensuring a more accurate analysis and visualization.</p><ul><li><strong>Min-max normalization</strong> adjusts the data to a specific range, typically between 0 and 1. It’s excellent when you want to maintain the original data distribution but need it on a smaller, standardized scale.<br><em>How does it work?</em><br>Let’s say you have MTTR ranging from 2 to 50 hours, and SLA adherence rates between 70% and 98%. Without normalization, a direct comparison of these metrics would yield unreliable insights due to the dissimilarity in their respective scales.<br>Min-max normalization adjusts these KPIs to a scale where the minimum value is transformed to 0, and the maximum value becomes 1. Every other value is adjusted proportionally in between.<br>For example, an MTTR of 2 hours becomes 0, an MTTR of 50 hours becomes 1, and an MTTR of 26 hours could be scaled to around 0.5. Similarly, an SLA adherence of 70% becomes 0, and 98% becomes 1. This scaling method allows you to make direct comparisons between disparate KPIs without losing the essence of what each KPI represents.</li><li><strong>Z-score standardization</strong> converts all data points into a common scale with an average of zero and standard deviation of one.<br><em>How does it work?</em><br>Let us take the example of Time to Resolve across different categories of incidents in your system. The Time to Resolve in minutes for network incidents might look like [30, 45, 40, 35, 50]. For application incidents, the times might be [70, 80, 75, 90, 85]. These two categories are on different scales, and comparing them directly could be misleading. Applying the <a href="https://corporatefinanceinstitute.com/resources/excel/z-score-standardize-function/#:~:text=A%20z%2Dscore%2C%20or%20standard,can%20be%20negative%20or%20positive.">Z-score standardization would calculate the Z-score</a> for each of the categories, helping you understand if your incident resolution times are deviating significantly from the norm in either of the categories.<br>While Min-Max is about rescaling, Z-score is about centering the data around the mean and considering the distribution. As a result, this normalization approach is considered suitable for algorithms that assume features are centered at zero and have similar variances.</li><li><strong>Decimal scaling</strong> is another method where data is moved by decimal places to bring all points into a similar range. It’s particularly effective when dealing with a wide range of values in a dataset.<br><em>How does it work?<br></em>This method standardizes the data but doesn’t change its distribution. For example, consider an organization receiving an average of 100 incident reports per day. To make it easier to analyze the data, the organization can decimal scale the number of incident reports by dividing it by 10. This means that the number of incident reports will now range from 1 to 10, instead of 100 to 1000.</li></ul><p>The approach helps to make the data more manageable and easier to analyze, ultimately helping track trends and identify areas where there are more incidents than others.</p><p>Before choosing one of the normalization methods, understand how they differ:</p><ul><li>Decimal scaling divides by a power of 10 to make values more manageable.</li><li>Min-max scales to [0,1] to make values comparable.</li><li>Z-score scales to mean 0 and std 1 to make values independent of distribution.</li></ul><p>Visualizing your KPI data can help you quickly spot patterns and anomalies that may be challenging to detect in raw data. The key, however, is choosing the right visual representation. It should align with the nature of your data and the insights you want to communicate. For instance, line graphs are excellent for tracking changes over time, while bar charts can effectively compare different categories.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*8qnCIejgVS_dRj5i.png" /><figcaption><em>Squadcast’s Incident Management Dashboard</em></figcaption></figure><p>And once your data is standardized, don’t just leave it as random numbers in a spreadsheet. Tools like <a href="https://www.squadcast.com/slo-tracker">Squadcast</a> can convert those raw figures into interactive charts and graphs, making it easier to spot <a href="https://www.squadcast.com/sre-best-practices/sla-vs-slo">Service Level Objective</a> (SLO) trends and patterns at a glance. Remember, your KPIs should inform your actions, and a clear, effective visualization is a crucial step in that process.</p><h3>Incident management KPI best practice #2: Leverage predictive analysis and AI-driven proactivity</h3><p>The ability to forecast potential incidents before they occur holds significant value. Techniques like <a href="https://en.wikipedia.org/wiki/Regression_analysis">regression analysis</a> or <a href="https://www.tableau.com/learn/articles/time-series-forecasting">time series forecasting</a> help achieve this by modeling relationships between variables and analyzing trends over time.</p><p>Consider leveraging AI/ML to automate KPI tracking. These technologies can churn massive amounts of data and find patterns otherwise practically impossible for a human to spot. More importantly, if Continual Service Improvement (CSI) is one of your key objectives, remember that the power of AI to learn and adapt over time can help your system become more intelligent and effective with every incident it encounters.</p><p>To support this, consider the following tips:</p><ul><li><strong>Create clear policies and procedures for data usage. </strong>Uniformity and clarity in the policies within an organization help streamline and supercharge the adoption of AI/ML.</li><li><strong>Ensure data quality</strong>. High-quality data serves as the foundation of predictive analysis. Make sure to understand the type of data collected and its application. Also, ensure your data is accurate, consistent, and relevant.</li><li><strong>Use the right tools. </strong>Utilize solutions like <a href="https://www.squadcast.com/incident-analytics-and-reliability-insights">Squadcast Analytics</a> to perform an extensive analysis of past incidents at both the broad organizational level and the granular team level. The platform further allows you to dive deeper to single out system bottlenecks that generate the most alerts and outages, categorizing these by teams, tags, and services.</li></ul><h3>Incident management KPI best practice #3: Embrace feedback loops and continuous learning</h3><p>When a KPI indicates a slowdown in incident resolution, the <em>call to action</em> is to essentially delve into it, understand the cause, and make the necessary adjustments. This feedback loop is essential for the continual refinement of your processes.</p><p>It is equally important that your team members are well-versed in interpreting KPIs. With each incident resolved, a new data set is added to the pool, providing additional opportunities to analyze and learn. Every cycle of this process represents an opportunity for improvement and a step closer to optimal efficiency and effectiveness.</p><p>While there is no thumb rule to promote this culture, consider adopting different strategies to enhance the continual learning environment:</p><ul><li>Use past incidents <a href="https://www.squadcast.com/blog/tips-to-make-your-retrospectives-meaningful">retrospectives</a> and data to create <em>what if</em>, hypothetical scenarios. Dry runs like these can help the team understand how different actions can influence KPIs and outcomes in real situations.</li><li>Involve your team in KPI development, understand why they’re tracking specific metrics, and how they can refine KPIs further. This can give them a deeper understanding of why certain metrics are tracked and how they can influence them.</li></ul><h3>Incident management KPI best practice #4: Create benchmarks and conduct performance assessments</h3><p>Compare KPIs with industry standards to see how your incident management stacks up against the competition. Benchmarking also lets you compare your incident management performance with best practices or historical data. This objective performance measurement can reveal your strengths and weaknesses, guiding your improvement efforts.</p><p>When interpreting benchmarks, your team size, resource allocation, and the complexity of incidents handled should all be factored in. It is also important to note that every organization has unique circumstances and goals. Ensure that industry averages are only seen as reference points, not absolute standards.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*tmHFkn2SFkkRumer.png" /><figcaption><em>Squadcast’s Service Level Objective (SLO) Dashboard</em></figcaption></figure><p>If real-time tracking of KPIs is crucial, utilize a dashboard like <a href="https://www.squadcast.com/slo-tracker">Squadcast’s Reliability Tracker</a> to provide an instant snapshot of current performance against set KPIs and benchmarks. Whether you opt for a commercial off-the-shelf solution or a custom-built one, ensure your dashboard can offer a snapshot of the current KPI performance versus the industry benchmarks.</p><figure><a href="https://register.squadcast.com/"><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dxg2KLcjq-n7d0LP4c8wTg.png" /></a></figure><h3>Conclusion</h3><p>Enterprises often misunderstand the true value of KPIs, seeing them merely as numeric markers. In contrast, KPIs help with strategic analysis by highlighting patterns, identifying bottlenecks, and guiding improvement areas.</p><p>But before you start monitoring KPIs, it is worth noting that although they provide crucial data, KPIs can’t always capture the intricacies of every operational nuance. This doesn’t undermine the value of KPIs, though. Instead, this emphasizes the need to supplement them with other factors — your team’s insights, situational understanding, the complexities of each incident, and a platform that helps you monitor your KPI performance efficiently.</p><p><em>Originally published at </em><a href="https://www.squadcast.com/incident-response-tools/incident-management-kpi"><em>https://www.squadcast.com</em></a><em> on September 18, 2023.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b1ad63e44cc6" width="1" height="1" alt=""><hr><p><a href="https://medium.com/squadcast/incident-management-kpi-best-practices-b1ad63e44cc6">Incident Management KPI Best Practices</a> was originally published in <a href="https://medium.com/squadcast">Squadcast</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>