<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Ben Abrams on Medium]]></title>
        <description><![CDATA[Stories by Ben Abrams on Medium]]></description>
        <link>https://medium.com/@majormoses?source=rss-84d426e4ad17------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*b2J9xQ0Cd0PEdYIp41tCew.png</url>
            <title>Stories by Ben Abrams on Medium</title>
            <link>https://medium.com/@majormoses?source=rss-84d426e4ad17------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 06:08:26 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@majormoses/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Contributing to Sensu: Updating Rubocop to resolve CVE-2017–8418]]></title>
            <link>https://medium.com/the-sensu-blog/contributing-to-sensu-updating-rubocop-to-resolve-cve-2017-8418-7b9a7bb8d9c?source=rss-84d426e4ad17------2</link>
            <guid isPermaLink="false">https://medium.com/p/7b9a7bb8d9c</guid>
            <category><![CDATA[ruby]]></category>
            <category><![CDATA[community]]></category>
            <category><![CDATA[monitoring]]></category>
            <category><![CDATA[sensus]]></category>
            <category><![CDATA[open-source]]></category>
            <dc:creator><![CDATA[Ben Abrams]]></dc:creator>
            <pubDate>Wed, 24 Jan 2018 20:17:46 GMT</pubDate>
            <atom:updated>2018-03-02T09:58:40.238Z</atom:updated>
            <content:encoded><![CDATA[<p><em>This is a guest post to the Sensu Blog from Ben Abrams, one of the lead maintainers of the Sensu Plugins organization. Learn more about the community at </em><a href="https://sensuapp.org/community"><em>sensuapp.org/community</em></a><em>.</em></p><p>Vulnerabilities are something that, when they come up, are important to resolve. One low priority one is in the form of CVE-2017–8418 and affects rubocop. Rubocop is used in every single (Ruby-based) Sensu Plugin, which means there are close to 200 repositories that need to be updated.</p><p>While this issue isn’t urgent, it is something I take to heart: I want everyone using Sensu to have safe and secure software running when they use a community plugin. I would also love your help to get us there.</p><p>On the most recent <a href="https://docs.google.com/document/d/1zXA-Qap4dARnB4bGutSvkyros4FWNqTiNlxzmtVguhQ/edit#heading=h.pw3b9ajw9wj2">Sensu Community Chat</a>, I walked listeners through updating the <a href="https://github.com/sensu-plugins/sensu-plugins-cassandra/">Cassandra Plugin</a> to resolving this vulnerability. Here is the full video, completely unscripted, on how to work through one of these updates on a repo:</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FMUzMgBTnPL0%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DMUzMgBTnPL0&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FMUzMgBTnPL0%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/bf5a416fa0ecf682c408151faffc1459/href">https://medium.com/media/bf5a416fa0ecf682c408151faffc1459/href</a></iframe><p>Here is more of a step-by-step guide below so you can follow along with links. I write up these notes in hope that you will setup your environment and go update one of the other <a href="https://github.com/sensu-plugins/community/issues/77">plugins that still needs it from this list</a>. It’d be great to have your contribution and I promise you will learn how to use Ruby a little bit better by doing so.</p><h3>Before you Begin</h3><p>I recommend you have the following installed to make your life much easier during this:</p><ul><li>A friendly text editor or IDE like VSCode or Atom (I use <a href="https://ide.atom.io/">Atom</a>)</li><li>Helpful integrations, like <a href="https://atom.io/packages/linter-rubocop">linter-rubocop</a> and <a href="https://atom.io/packages/rubocop-auto-correct">rubocop-auto-correct</a></li><li>GitHub configured, either through <a href="https://desktop.github.com/">the Desktop App</a> or <a href="https://help.github.com/articles/set-up-git/#setting-up-git">command-line</a></li></ul><p>Having these configured will make everything else a lot easier.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ll0907cew7dgut-_uyAWoQ.png" /><figcaption>The “meta” issue that tracks updates across our many plugins.</figcaption></figure><h3>(1) Claim a plugin</h3><p>Go to the meta-issue that explains the need across the community: <a href="https://github.com/sensu-plugins/community/issues/77">https://github.com/sensu-plugins/community/issues/77</a></p><p>Find an unchecked box next to a plugin, then scroll to the very bottom and add a comment saying you’ll cover it (making sure someone else hasn’t claimed it along the way):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pLH4-3tnjld3gXwzDD53oQ.png" /></figure><h3>(2) Fork and clone that plugin</h3><p>Click the fork button to make your own remote copy of the repository, then clone it locally. So assuming I’m working on the Cassandra plugin, I would:</p><ul><li>Go to <a href="https://github.com/sensu-plugins/sensu-plugins-cassandra">https://github.com/sensu-plugins/sensu-plugins-cassandra</a></li><li>Click the Fork button in the top right corner</li><li>Clone it locally either using the GitHub Desktop or by running git clone</li><li>Switch to a new branch, preferably fix/CVE-2017-8418 for your code</li></ul><p>Now it’s ready to open in your IDE.</p><h3>(3) Pin to the new rubocop version</h3><p>Updating rubocop is as easy as updating the .gemspec file in the directory. You will see a line that looks like:</p><pre>s.add_development_dependency &#39;rubocop&#39;, &#39;~&gt; 0.41.0&#39;</pre><p>And update it to:</p><pre>s.add_development_dependency &#39;rubocop&#39;, &#39;~&gt; 0.51.0&#39;</pre><p>And run bundle to install the new version:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/828/1*Jl4l99iCsJ8n9sG43Hy5yA.gif" /></figure><p>We’re using rubocop 0.51.0 across all repositories, even though newer versions have come out since then, so everyone can have the same version of rubocop across the board. It makes maintaining plugins easier as a user of Sensu.</p><h3>(4) Resolve rubocop linting violations</h3><p>Now that rubocop is updated, it will include new rules. If you open up your IDE with the recommended plugins installed (rubocop linter and autocorrect), you’ll be able to work through these quickly. A few tips to do so:</p><ul><li>Use # rubocop:disable $THING where $THING is the rubocop violation</li><li>Don’t use# rubocop:disable all but stick with the above</li><li>Never add disable all to the top of a file please</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Vursg4WdWqWJRd1JjtwdUw.png" /><figcaption>Using a specific disable</figcaption></figure><p>EDIT (03/02/2018): You should not change any code to use .match?(/SOME_REGEX/) as it does not exist in ruby &lt; 2.4 you can add this to the rubocop.yml:</p><pre># .match?() only exists in ruby 2.4<br>Performance/RegexpMatch:<br>  Enabled: false</pre><h3>(5) Update the Changelog</h3><p>We want to make sure other users of this plugin know what is in the next release. To do so, we ask that you update the CHANGELOG.md file. These changes will go under the ### Security section of ## Unreleased and a maintainer will move it to the next release section. Here’s an <a href="https://github.com/sensu-plugins/sensu-plugins-cassandra/pull/10/files#diff-4ac32a78649ca5bdd8e0ba38b7006a1e">example from this PR</a>:</p><pre>## [Unreleased]</pre><pre>### Security<br>- updated `rubocop` dependency to `~&gt; 0.51.0` per: https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2017-8418. (@majormoses)<br><br>### Breaking Changes<br>- in order to bring in new rubocop dependency we need to drop ruby 2.0 support as it is EOL and aligns with out support policy. (@majormoses)</pre><p>Feel free to copy and paste this and replace your username.</p><h3>(6) Open the Pull Request and participate in review</h3><p>Once you’ve committed your changes to your local branch and pushed them to your remote fork, you can open a Pull Request.</p><p>The quick way to do this is to navigate to the plugin you updated, like <a href="https://github.com/sensu-plugins/sensu-plugins-cassandra/">sensu-plugins/sensu-plugins-cassandra</a>, and click the prompt.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mZjlGBnnwJ5sn2U6500_UA.png" /><figcaption>A similar prompt to this will pop up for you.</figcaption></figure><p>At this point a maintainer will get engaged and may ask you changes or feedback. Please keep in mind this is done with the understanding that no one is perfect, especially me, and we will improve this code together. To make sure you don’t miss any updates, be sure to <strong>subscribe to notifications</strong> on the right-hand side of the Pull Request page.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/564/1*JXOXnPl6TUSJv16n4wnI-g.png" /></figure><h3>(7) Merge and release (Maintainers step)</h3><p>Once your code receives a 👍 from a maintainer, that maintainer will merge and cut a release. You can <a href="https://youtu.be/MUzMgBTnPL0?t=45m48s">see that process here</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xOui6znPe4WP5nHcCscecg.png" /><figcaption>Find the latest version <a href="https://github.com/sensu-plugins/sensu-plugins-cassandra/releases">under the Releases tab</a></figcaption></figure><p>Once complete, you can go to the Releases tab on that plugin and download your contribution!</p><h3>Rinse and Repeat</h3><p>By following these (rough) steps you will make a very helpful contribution to the Sensu Community and learn a lot about the GitHub workflow. Along the way you will also learn Rubocop’s linting requirements, which have taught me to be a much better developer.</p><p>I hope after you finish your first plugin you will start on your second! There’s a long list and a lot to learn. Slowly we will work through this together. As always, if you need help in anyway, tag me on GitHub @majormoses or reach out on <a href="http://slack.sensu.io/">Sensu Community Slack</a>.</p><p><em>This is a guest post to the Sensu Blog from Ben Abrams, one of the lead maintainers of the Sensu Plugins organization. Learn more about the community at </em><a href="https://sensuapp.org/community"><em>sensuapp.org/community</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7b9a7bb8d9c" width="1" height="1" alt=""><hr><p><a href="https://medium.com/the-sensu-blog/contributing-to-sensu-updating-rubocop-to-resolve-cve-2017-8418-7b9a7bb8d9c">Contributing to Sensu: Updating Rubocop to resolve CVE-2017–8418</a> was originally published in <a href="https://medium.com/the-sensu-blog">The Sensu blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Writing Sensu Plugin Tests with Test-Kitchen and Serverspec]]></title>
            <link>https://medium.com/the-sensu-blog/writing-sensu-plugin-tests-with-test-kitchen-and-serverspec-b646d2eeee51?source=rss-84d426e4ad17------2</link>
            <guid isPermaLink="false">https://medium.com/p/b646d2eeee51</guid>
            <category><![CDATA[ruby]]></category>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[community]]></category>
            <category><![CDATA[monitoring]]></category>
            <category><![CDATA[software-testing]]></category>
            <dc:creator><![CDATA[Ben Abrams]]></dc:creator>
            <pubDate>Thu, 07 Dec 2017 16:21:01 GMT</pubDate>
            <atom:updated>2017-12-13T16:19:23.695Z</atom:updated>
            <content:encoded><![CDATA[<p><em>This is a guest post to the Sensu Blog from Ben Abrams, one of the lead maintainers of the Sensu Plugins organization. He offered to share his thoughts in his own words, which you can do too by emailing </em><a href="/service/mailto:community@sensu.io"><em>community@sensu.io</em></a><em>. Learn more about the community at </em><a href="https://sensuapp.org/community"><em>sensuapp.org/community</em></a><em>.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*cIblp54Www0kWjqooiar2w.gif" /><figcaption>It’s time to get into the (test) kitchen ready for testing</figcaption></figure><p><a href="https://blog.sensuapp.org/our-approach-to-testing-sensu-plugins-869260eba94c">In my last post</a> I talked a lot about why I love testing infrastructure and the progress we have made in the Sensu community to increase our test coverage. Now I want to walk you through exactly how to setup your environment from scratch to be able to do so.</p><h3>Start Small</h3><p>Before we get too far in, I want to remind you to start small. Rather than going OMG test all the things! Too much work! Overwhelmed! Take a step back and decide the most effective type of testing. My simple heuristic is: unit tests for libraries, integration tests for plugins. Once you have decided on the type of testing you should look for what are the most critical things to test. Choose one thing and test it really well. Over time the testing suite will grow and it will become quicker to get through code review as maintainers will be more confident merging something that has a test that passes.</p><h3>Install dependencies</h3><p>For plugins we have standardized our integration testing around the following tools:</p><p><strong>Test-kitchen:</strong> Provides a framework for developing and testing infrastructure code and software on isolated platforms</p><ul><li><a href="https://github.com/test-kitchen/test-kitchen">https://github.com/test-kitchen/test-kitchen</a></li><li><a href="https://github.com/test-kitchen/test-kitchen/wiki/Getting-Started">https://github.com/test-kitchen/test-kitchen/wiki/Getting-Started</a></li><li><a href="https://docs.chef.io/kitchen.html">https://docs.chef.io/kitchen.html</a></li><li><a href="https://docs.chef.io/config_yml_kitchen.html">https://docs.chef.io/config_yml_kitchen.html</a></li><li><a href="http://kitchen.ci/">http://kitchen.ci/</a></li></ul><p><strong>Kitchen-docker: </strong>Docker driver for test-kitchen to allow using in a lighter weight fashion that traditional virtualization such as vagrant + virtualbox.</p><ul><li><a href="https://github.com/test-kitchen/kitchen-docker">https://github.com/test-kitchen/kitchen-docker</a></li></ul><p><strong>Serverspec:</strong> The verification platform. It is an expansion on rspec that allows you to write very simple tests.</p><ul><li><a href="http://serverspec.org/">http://serverspec.org/</a></li><li><a href="http://serverspec.org/resource_types.html">http://serverspec.org/resource_types.html</a></li><li><a href="http://serverspec.org/advanced_tips.html">http://serverspec.org/advanced_tips.html</a></li></ul><h3>Setting up your framework</h3><p>Basically there are a few key components and once you have the required gems installed you will want to start creating a skeleton. You will want to decide the following:</p><ul><li>What platforms do I want to test? For example debian, centos, windows, etc</li><li>What versions of languages do I want to test? For example ruby 2.1, 2.2, 2.3.0, 2.4.0</li><li>What driver you will use for test-kitchen? For example kitchen-docker (preferred), kitchen-vagrant (a bit heavy but super helpful for certain scenarios), kitchen-dokken (a pretty reasonable solutions when using process supervisors), kitchen-ec2 (convenient but costs money), etc</li></ul><p>For this example we are gonna test on Debian with several versions of Ruby. This looks something like this in your .kitchen.yml:</p><p><a href="https://github.com/sensu-plugins/sensu-plugins-dns/blob/1.3.0/.kitchen.yml#L1-L31">https://github.com/sensu-plugins/sensu-plugins-dns/blob/1.3.0/.kitchen.yml#L1-L31</a></p><h4>.kitchen.yml configuration</h4><p>This file has a few essential parts.</p><h4>Driver</h4><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/008cf2b8737e04ab663370010ca55f0a/href">https://medium.com/media/008cf2b8737e04ab663370010ca55f0a/href</a></iframe><p>This is telling kitchen to use the docker driver without the need of sudo to do so. There might be scenarios where you need to use other drivers such as kitchen-dokkenwith sudo as true when say interacting with systemd.</p><h4>Provisioner</h4><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4b56ab58ce3949929292761b6389b73b/href">https://medium.com/media/4b56ab58ce3949929292761b6389b73b/href</a></iframe><p>This tells us that we are gonna use a shell provisioner (rather than say chef, puppet, ansible, saltstack, etc) and where to find it.</p><h4>Verifier</h4><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/dc846dd30259e686ab5a5c6eea6a7099/href">https://medium.com/media/dc846dd30259e686ab5a5c6eea6a7099/href</a></iframe><p>This tells kitchen where to find the verifier when using inspec you can omit I believe.</p><h4>Platforms</h4><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/8ff87cb3e56da9e74539777243f1d80e/href">https://medium.com/media/8ff87cb3e56da9e74539777243f1d80e/href</a></iframe><p>This tells kitchen to use debian 8as a platform, you could include ubuntu, centos, etc with multiple versions.</p><h4>Suites</h4><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5bc7752e1c6a5fc28ffd2f91a0bcf8b0/href">https://medium.com/media/5bc7752e1c6a5fc28ffd2f91a0bcf8b0/href</a></iframe><p>Here we tell kitchen which suites to run which will create a matrix with the platforms desired. This is also telling the driver to use a specific image. In this case we are using docker and want the ruby-slim containers with the appropriate version to test. Therefore the suites are versions of ruby.</p><h4>Test directory Structure</h4><p>Under the repository root we expect all the testing artifacts, etc to go in ./test</p><p>Here is a snapshot of what the directory structure looks like:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/55678df2f38383b698073a29628d2dc8/href">https://medium.com/media/55678df2f38383b698073a29628d2dc8/href</a></iframe><p>Under ./test/integration directory we helpers and specific suites. As we want to run all the tests on all suites we write the tests in ./test/helpers/serverspec/ and in the other directories just include the specs. If you wanted to write a specific check for a specific ruby version (say in ruby 2.4.1 they require a new gem to be installed) you can add that to a new spec under the appropriate folder.</p><h3>Actually writing a test</h3><p>A really simple example of a test would be does the script execute, return the right exit code, and output matches expected.</p><p>An example for DNS:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/6853ab97cc98dd84fb5ecc1a42a3744e/href">https://medium.com/media/6853ab97cc98dd84fb5ecc1a42a3744e/href</a></iframe><p>This tells us that we executed the check, and it resolved the DNS entry and returned that it is healthy. A test look <a href="https://travis-ci.org/sensu-plugins/sensu-plugins-dns/jobs/297261015#L1550-L1554">something like this</a>.</p><p>A negative test to make sure that things fail when you improperly configure them you might try passing without a domain like this:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e8f9c5ca4cf408c065fddafe934d815f/href">https://medium.com/media/e8f9c5ca4cf408c065fddafe934d815f/href</a></iframe><p>Which would result in a <a href="https://travis-ci.org/sensu-plugins/sensu-plugins-dns/jobs/297261015#L1574-L1578">status like this in Travis</a>. Those two things tells us that at least the very basic DNS checking is working.</p><h4>Running Tests and verification</h4><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/76efcc7f1eb1f64701788b3b919339d9/href">https://medium.com/media/76efcc7f1eb1f64701788b3b919339d9/href</a></iframe><p>Say you are seeing a failure on Travis on ruby 2.2 but not on others you can run just the 2.2 verification tests like this:</p><pre>$ bundle exec kitchen test ruby-22-debian-8</pre><p>You will see a bunch of output which I have removed here but you can see the tests executed:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2d49a60534c219abbe466d69dca41ed6/href">https://medium.com/media/2d49a60534c219abbe466d69dca41ed6/href</a></iframe><h4>Integrate to CI (Travis)</h4><p>There are a few conveniences we like to have for our continuous integration.</p><h4><em>Rakefile</em></h4><p>Add some simple tasks to make life easier:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/42dc00d6f1d08551e7375353d88c41cd/href">https://medium.com/media/42dc00d6f1d08551e7375353d88c41cd/href</a></iframe><p>So a couple things to note here:</p><ul><li>Pull in standard kitchen rake tasks which makes life easy for development</li><li>Define a quick task that has all but integration testing. The reason for this is that we want to by default run all tests but in travis (or any CI for that matter) we use <a href="https://docs.travis-ci.com/user/customizing-the-build#Build-Matrix">matrix jobs</a> to run tests in parallel speeding up the amount of time it takes for testing to complete in. Therefore we will use quick task plus some extra bash glue to run the right version later in travis.</li></ul><p>See <a href="https://github.com/sensu-plugins/sensu-plugins-dcos/blob/0.4.0/Rakefile">this Rakefile</a> for a full example.</p><h4><em>.travis.yml</em></h4><p>Configuring the configuration file for Travis CI has a lot of important parts to it.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/ef17195b7217a3cb12cbfab489f7fde3/href">https://medium.com/media/ef17195b7217a3cb12cbfab489f7fde3/href</a></iframe><p>Breaking down the important bits:</p><ul><li>We tell travis that this is a ruby project and to test the following versions of ruby: 2.1, 2.2, 2.3.0, 2.4.1</li><li>We do some docker setup</li><li>Run the quick rake tests (all but integration)</li><li>Run the kitchen test for the platforms of choice on the version of ruby that travis is testing</li><li>Build and install gem</li><li>Deploy to rubygems.org with a key, <a href="https://docs.travis-ci.com/user/encryption-keys/">see encrypting variables in travis</a> on tagged commits, testing with the same ruby versions as previously tested in branches and PRs.</li></ul><p>For a full example see, review <a href="https://github.com/sensu-plugins/sensu-plugins-dcos/blob/0.4.0/.travis.yml">this Travis CI config file</a>.</p><h3>Expand &amp; Profit</h3><p>Once you have the most critical tests written and you have the framework to do so in place you can quickly add tests and when you get the appropriate coverage you will notice that the code becomes more stable. I can tell you at my job we have pretty great chef testing using 99% of the same exact tools outlined here. It has given us a great degree of confidence to do gnarly refactors safely, introduce new features without breaking old ones, and that we are not being broken by upstream changes. I have seen at least 4 potential breakages (and one MAJOR outage) in the last month that were prevented by having great testing and integrating it with CI.</p><p>You will also know that your PRs will get a lot faster on reviews because as a maintainer I am more likely to scrutinize change when I can’t see that it is actually working with 0 effort.</p><p>I hope you will get more involved in the effort! Contribute tests to <a href="https://github.com/sensu-plugins">your favorite plugins</a> and <a href="http://slack.sensu.io/">join us on Slack</a> to talk about it with fellow maintainers. We would love to get you involved.</p><p><em>This is a guest post to the Sensu Blog from Ben Abrams, one of the lead maintainers of the Sensu Plugins organization. He offered to share his thoughts in his own words, which you can do too by emailing </em><a href="/service/mailto:community@sensu.io"><em>community@sensu.io</em></a><em>. Learn more about the community at </em><a href="https://sensuapp.org/community"><em>sensuapp.org/community</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b646d2eeee51" width="1" height="1" alt=""><hr><p><a href="https://medium.com/the-sensu-blog/writing-sensu-plugin-tests-with-test-kitchen-and-serverspec-b646d2eeee51">Writing Sensu Plugin Tests with Test-Kitchen and Serverspec</a> was originally published in <a href="https://medium.com/the-sensu-blog">The Sensu blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Our Approach to Testing Sensu Plugins]]></title>
            <link>https://medium.com/the-sensu-blog/our-approach-to-testing-sensu-plugins-869260eba94c?source=rss-84d426e4ad17------2</link>
            <guid isPermaLink="false">https://medium.com/p/869260eba94c</guid>
            <category><![CDATA[testing]]></category>
            <category><![CDATA[community]]></category>
            <category><![CDATA[monitoringlove]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[monitoring]]></category>
            <dc:creator><![CDATA[Ben Abrams]]></dc:creator>
            <pubDate>Thu, 07 Dec 2017 16:16:01 GMT</pubDate>
            <atom:updated>2017-12-15T20:01:03.555Z</atom:updated>
            <content:encoded><![CDATA[<p><em>This is a guest post to the Sensu Blog from Ben Abrams, one of the lead maintainers of the Sensu Plugins organization. He offered to share his thoughts in his own words, which you can do too by emailing </em><a href="/service/mailto:community@sensu.io"><em>community@sensu.io</em></a><em>. Learn more about the community at </em><a href="https://sensuapp.org/community"><em>sensuapp.org/community</em></a><em>.</em></p><h3>The Power of Tests</h3><p>I have the benefit building great testing in my organization and the peace of mind that gives us. It prevents bugs not only in our own code, but also helps find issues with how it interacts with upstream projects.</p><p>I’ve been a maintainer of Sensu’s Plugins since <a href="https://github.com/sensu-plugins/community/issues/17">May 2017</a> and I could not be happier to get to the point of sharing my philosophy on testing. The maintainers have started to roll out these practices in a standardized way, so I write this as a means to sharing <em>the why</em> behind the work. I will follow up with a more in depth <em>how</em> in a future post.</p><p>If you love Sensu Plugins and know a little Ruby, I hope you’ll join us on this journey to bring tests to all our plugins.</p><h3>Why do I care? (TL;DR)</h3><p>Ruby-based plugins in the Sensu community now have a pattern to integration testing that are rolling out across all maintained plugins <a href="https://github.com/sensu-plugins/community/issues/41">with your help</a>. This testing is essential to allow us to iterate quickly and know we’re not introducing bugs to code we all rely on. I’m writing up our community testing practices here to give us a solid foundation going forward.</p><h4>Refactor and improve safely</h4><p>The most essential benefit of a test is that it tells us when functionality changes. I want to improve plugin tests so I know a proposed contribution (PR) adding new features or changing existing features will not break existing use cases. When code does break, and it will break, I know it will either mean the PR needs updates or we have a breaking change and need to update the tests and `CHANGELOG.md` to acknowledge that.</p><p>When adding a new feature you should consider the following:</p><ul><li><em>How do I know this works as expected?</em></li><li><em>Can I write an automated test for it?</em></li><li><em>If I can’t write an automated test, can I give manual instructions to someone to verify it works?</em></li></ul><h4>Test Artifacts for Accepting Code into the Community Projects</h4><p>We have gone through some pretty radical changes over the last year or so with the plugins organization. The organization has grown in popularity <a href="https://blog.sensuapp.org/introducing-sensu-inc-9f1d2027c577">as Sensu Inc became a company</a>, but it may not be well known that it’s 99% maintained by community members. There are now nearly 200 plugins and extensions covering a lot of tech and various products not all maintainers will have heard of, let alone the expertise to review suggestions SMEs are proposing. We rely heavily on the code review and testing to give us confidence that merging code will not break another feature.</p><p>That is why, earlier this year, we updated our pull request templates to review a testing artifact for most code contributions. This slight change to process has helped assure a stable code base even in the absence of automated tests. That focus on quality plugins for the community over some shiny new feature that will break other users has done our community well and will continue to be our policy. Code is only merged when it works for most cases, not just yours.</p><h3>How We’re Approaching Types of Testing</h3><p>We all know we need tests. Now the big question: what <em>kind </em>of test is right for my code?</p><h4>Manual</h4><p>This process has done us well for the last 6 months. The benefit is that it is the easiest option given that it does not require writing code. The difficult here is that every minor edit during a code review requires someone to re-run these “tests,” making it not terribly scalable. There might be cases where you might need to manually verify a few things but ideally the testing should be automated so that when you are contributing changes they are automatically run and give you feedback when you do break something.</p><p>As this is a pretty big area to cover I think it would be best to <a href="https://github.com/sensu-plugins/community/blob/master/PULL_REQUEST_PROCESS.md#7-testing-artifacts.">review examples of such on the Pull Request template</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/911/0*s5xlanKl_ELZPfd-." /></figure><h4>Automated</h4><p>Automated tests is where we’re making great progress these days. The practice of automating tests comes in a few forms.</p><h4>Linting</h4><p>Linting is a huge advantage when I think about the scale of the Sensu Plugins projects. We want to make sure that we follow the best practices for each programming language so we can rely on that as an early validation of code quality. For Ruby we use rubocop and in Python we run pep8/flake8. You can configure rules in a `.rubocop.yml` or `.flake8rc` respectively in each repository as it makes sense. I think of linters as the laws of the land if you are new to the language. They prevent us from pitfalls and become better developers. The linter rules are designed to be followed, so please follow them. Occasionally, you come across an overkill of a rule that’s worth disabling, but please open a PR to discuss it with a maintainer and see if they are ok with removing it.</p><h4>Unit Tests</h4><p>A beneficial form of testing, unit tests are typically written against library functions. These offer a lot of value on libraries where writing an end-to-end integration testing may be difficult. All libraries should have unit tests, but not necessarily plugins.</p><p>Examples:</p><ul><li><a href="https://github.com/sensu-plugins/sensu-plugin/blob/v2.1.0/test/deep_merge_test.rb">https://github.com/sensu-plugins/sensu-plugin/blob/v2.1.0/test/deep_merge_test.rb</a></li><li><a href="https://github.com/sensu-plugins/sensu-plugin/blob/v2.1.0/test/mutator_test.rb">https://github.com/sensu-plugins/sensu-plugin/blob/v2.1.0/test/mutator_test.rb</a></li></ul><h4>Integration Tests</h4><p>These are the holy grail of testing, especially for plugins. Integration tests not only does exercise the code as designed, but it pulls in dependencies to test the integration of the two or more required components that it depends on. These are the make-or-break tests for plugin features and should be a major focus of our community!</p><h4>Diving In: Anatomy of an Integration Test</h4><p>Bootstrap dependencies: This typically requires bootstrapping your test with dependencies. This includes (but is not limited to):<br>Dev/Build tools such as a c compiler, system libraries, linters, testing frameworks, etc.</p><p>Examples:</p><ul><li><a href="https://github.com/sensu-plugins/sensu-plugins-postgres/blob/47caab71f0a2f858733a50ac975d7d18d2660944/.travis.yml#L8">https://github.com/sensu-plugins/sensu-plugins-postgres/blob/47caab71f0a2f858733a50ac975d7d18d2660944/.travis.yml#L8</a></li><li><a href="https://github.com/sensu-plugins/sensu-plugins-dns/blob/1.3.0/test/fixtures/bootstrap.sh#L5">https://github.com/sensu-plugins/sensu-plugins-dns/blob/1.3.0/test/fixtures/bootstrap.sh#L5</a></li></ul><p>Dependent systems such as datastores, web servers, etc. For example:</p><ul><li><a href="https://github.com/sensu-plugins/sensu-plugins-postgres/blob/47caab71f0a2f858733a50ac975d7d18d2660944/.travis.yml#L22-L23">https://github.com/sensu-plugins/sensu-plugins-postgres/blob/47caab71f0a2f858733a50ac975d7d18d2660944/.travis.yml#L22-L23</a></li><li><a href="https://github.com/sensu-plugins/sensu-plugins-redis/blob/2.3.0/test/fixtures/bootstrap.sh#L20-L49">https://github.com/sensu-plugins/sensu-plugins-redis/blob/2.3.0/test/fixtures/bootstrap.sh#L20-L49</a></li></ul><h4>Mocks or configure services</h4><p>In some cases it may not be reasonable to actually say run a full DCOS mesos cluster with multiple containers running. In this case we decide to mock the required tests with a web server such as nginx as the DCOS api is over http and those are fairly simple to mock in most cases.</p><p>Example:</p><ul><li><a href="https://github.com/sensu-plugins/sensu-plugins-dcos/blob/0.4.0/test/fixtures/bootstrap.sh#L34-L140">https://github.com/sensu-plugins/sensu-plugins-dcos/blob/0.4.0/test/fixtures/bootstrap.sh#L34-L140</a></li></ul><h4>Write tests against integration services</h4><p>There are essentially two types of verification tests: positive and negative verification.</p><p>Examples:</p><ul><li><a href="https://github.com/sensu-plugins/sensu-plugins-dcos/blob/0.4.0/test/integration/helpers/serverspec/check-dcos-ping-shared_spec.rb">https://github.com/sensu-plugins/sensu-plugins-dcos/blob/0.4.0/test/integration/helpers/serverspec/check-dcos-ping-shared_spec.rb</a><br><a href="https://github.com/sensu-plugins/sensu-plugins-dcos/blob/0.4.0/test/integration/helpers/serverspec/check-dcos-component-health-shared_spec.rb">https://github.com/sensu-plugins/sensu-plugins-dcos/blob/0.4.0/test/integration/helpers/serverspec/check-dcos-component-health-shared_spec.rb</a></li><li><a href="https://github.com/sensu-plugins/sensu-plugins-dns/blob/1.3.0/test/integration/helpers/serverspec/check-dns-shared_spec.rb">https://github.com/sensu-plugins/sensu-plugins-dns/blob/1.3.0/test/integration/helpers/serverspec/check-dns-shared_spec.rb</a><br><a href="https://github.com/sensu-plugins/sensu-plugins-redis/blob/2.3.0/test/integration/helpers/serverspec/check-redis-ping_spec.rb">https://github.com/sensu-plugins/sensu-plugins-redis/blob/2.3.0/test/integration/helpers/serverspec/check-redis-ping_spec.rb</a></li></ul><p>Write Verification Tests</p><p>Positive tests: if I do <em>this,</em> I expect it to work.</p><p>Examples:</p><ul><li><a href="https://github.com/sensu-plugins/sensu-plugins-dns/blob/1.3.0/test/integration/helpers/serverspec/check-dns-shared_spec.rb#L15-L18">https://github.com/sensu-plugins/sensu-plugins-dns/blob/1.3.0/test/integration/helpers/serverspec/check-dns-shared_spec.rb#L15-L18</a></li><li><a href="https://github.com/sensu-plugins/sensu-plugins-dcos/blob/master/test/integration/helpers/serverspec/check-dcos-component-health-shared_spec.rb#L14-L17">https://github.com/sensu-plugins/sensu-plugins-dcos/blob/master/test/integration/helpers/serverspec/check-dcos-component-health-shared_spec.rb#L14-L17</a></li></ul><p>Negative tests can be just as powerful. They ask the other question: if I improperly use this, it will <em>error</em>. These are also valid to verify critical or warning events.</p><p>Examples:</p><ul><li><a href="https://github.com/sensu-plugins/sensu-plugins-dns/blob/1.3.0/test/integration/helpers/serverspec/check-dns-shared_spec.rb#L35-L38">https://github.com/sensu-plugins/sensu-plugins-dns/blob/1.3.0/test/integration/helpers/serverspec/check-dns-shared_spec.rb#L35-L38</a></li><li><a href="https://github.com/sensu-plugins/sensu-plugins-dns/blob/1.3.0/test/integration/helpers/serverspec/check-dns-shared_spec.rb#L40-L43">https://github.com/sensu-plugins/sensu-plugins-dns/blob/1.3.0/test/integration/helpers/serverspec/check-dns-shared_spec.rb#L40-L43</a></li><li><a href="https://github.com/sensu-plugins/sensu-plugins-dcos/blob/master/test/integration/helpers/serverspec/check-dcos-component-health-shared_spec.rb#L29-L32">https://github.com/sensu-plugins/sensu-plugins-dcos/blob/master/test/integration/helpers/serverspec/check-dcos-component-health-shared_spec.rb#L29-L32</a></li></ul><h3>Testing of Sensu Plugins</h3><p>We’re at a point where Sensu Plugins are getting the test coverage that will give all of us more confidence on every PR review. Relying on manual testing was a great first step for us, but the move to automated unit tests for libraries and integration tests for plugins will be a huge leap forward. As a maintainer, that makes me excited for where we can go from here. As a Sensu user, that makes my job a little easier.</p><blockquote>Relying on manual testing was a great first step for us, but the move to automated unit tests for libraries and integration tests for plugins will be a huge leap forward.</blockquote><p>I hope you will get involved in the effort! Contribute tests to your favorite plugins and <a href="http://slack.sensu.io/">join us on Slack</a> to talk about it with fellow maintainers. We would love to get you involved.</p><p><em>This is a guest post to the Sensu Blog from Ben Abrams, one of the lead maintainers of the Sensu Plugins organization. He offered to share his thoughts in his own words, which you can do too by emailing </em><a href="/service/mailto:community@sensu.io"><em>community@sensu.io</em></a><em>. Learn more about the community at </em><a href="https://sensuapp.org/community"><em>sensuapp.org/community</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=869260eba94c" width="1" height="1" alt=""><hr><p><a href="https://medium.com/the-sensu-blog/our-approach-to-testing-sensu-plugins-869260eba94c">Our Approach to Testing Sensu Plugins</a> was originally published in <a href="https://medium.com/the-sensu-blog">The Sensu blog</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>