On the move
A software testing exercise
Here’s a familiar sort of problem. You’re moving something, or things, from one place (A) to another (B), where the state of the thing (or things) at both departure (A) and arrival (B) matter. You’re relocating, let’s say, and naturally you want all of your stuff not just to arrive at their new home but also to arrive safely and generally intact. Or maybe you’re a distributor for a grocery chain and want to ensure that the right stuff arrives at the right places on time and in good shape.
Get Paul Martin’s stories in your inbox
Join Medium for free to get updates from this writer.
This sort of problem also arises within software development. At a very general level, it’s among its most central problems; it arises every time we take all of the changes that have been made during some large or small stretch of product development — the last month or day or twenty minutes — and put it through whatever it takes to turn those changes (a “build process”) into something (a “build”) that can be run or deployed or shipped. We know what changes we’ve made; once built, do they have the desired or intended effect? Do those changes actually produce the value we mean to provide?
A variation of this problem arises when deploying a build to a new or different environment. The risks in this case may result as much from differences in the environments as the deployment process itself.
There are many more specific cases. Here’s a small one to chew on. You are a software tester working on a web application that includes (among many other things) a bunch of web pages the content for which is produced and “owned” by a marketing or communications team outside of the engineering team. In addition to where in the web application’s URL structure these pages live, they each have specific content, in multiple languages, as well as contain links to other pages in the site (“internal” links) as well as links to other sites outside of it (“external” links).
Let us further suppose that the mechanism for building and deploying these pages is different than that used for building or deploying the web application as such. By making environment-specific changes on top of content stored in a separate content management system, the marketing or communications team can control and push content changes out of band of engineering schedules. (This effectively collapses the build and deployment “movements” in figures 1 and 2 into one.)
Finally, imagine that this marketing or communications team is responsible for the state of these pages on your main test environment, which we may take as our location A. And so as this content is “moved” to locations (environments) B, C,…, your problem is to measure whether the state of these moved pages matches their state on location A. While offhand this might seem a pretty uninteresting problem, technically or otherwise, the page content involved might provide highly valuable information, potentially to millions of customers or users.
So then, what kind of things would you test and why?
