Choosing Test Cases

Preamble: I am currently undergoing my Masters of Data Science and I am taking two preliminary courses in Python and R. I will be documenting journey in a series of journals that I will post on Medium

Image Source: Real Python

Testing is essentially just as important as coding itself. It makes no sense to write a bunch of code without any iota of correctness. Matter of fact, in most companies — most software development team would not accept an individual or team code unless it has gone through a thorough testing suite.

Let’s look a-bit closely at choosing test cases for a test suite

Example:

Assume the max function does not exist and we created a function that pull the max integer in a list. Suppose we created a test for the function using the following use cases and it passes

Image Source: University of Toronto

Would you be confident on the strength of your functions and its test? What if it passed 20 of the same use cases ? 50? 100? 250? — Would this make you more confident? Your answer should be skeptical. Yes, these are a-lot of tests but how do we know we have not omitted a possible scenario that could cause a failure? How do we know that we have included every possible scenario?

The fundamental dilemma is that we want to ensure that the function works in all cases but that is too many cases to test for. The Venn diagram below visualizes this

Image Source: University of Toronto

Make a Convincing Argument

The analogy we are going to use next will sound familiar to sampling.

1. We will split up all the possible use cases (or call) in categories
2. Pick a representative use case (or call) in each category

Sounds like Sampling huh😉

Get Stephen Ayeni’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

Look at our Venn Diagram now:

Image Source: University of Toronto

If we choose our categories well, it is logical to extrapolate the results of the test of one representative use case (or call) to other potential use cases (or calls) in the same category

This wil transform our graph into

Image Source: University of Toronto

Now for each use case, we either have test or untested but reasonable to extrapolate from category representative

How to choose relevant properties

Now, how do we go about choosing the relevant categories — Just like we said in the earlier blogpost, we base the categories on the properties of the inputs. Here are some properties and values for them:

1. The size of an object (str, int, list etc.): 0, 1, small, large, even, odd
2. The position of the value in an ordered sequence (tuple, list etc.): Beginning, Ending, Elsewhere
3. The relative position of two objects in an ordered sequence: Adjacent, separated
4. The presence of duplicate: yes, no
5. Ordering: unsorted, sorted, non-decreasing, non-increasing
6. The value of an integer: 0,1, positive, negative, even, odd, small, large
7. The value of string: alphanumeric only, special characters only
8. The location of whitespace in a string: Beginning, Middle, Ending, Multiple whitespaces, adjacent whitespace, multiple adjacent white space, different white space.
9. and more depending on the parameter of the functions, there could be many other properties.

Fundamentally, you have to try to break the code and if you can’t break it — you might be cooking something special. You have a pretty good argument.