Improving Semantics and Correctness · Illustrated via Redux State

Abstract

JavaScript has built-in atomic scalar types, such as numbers and booleans. It can also represent composite product types via arrays or record types via objects. However, it lacks an immediate solution for disjoint sum types. Sum types (a.k.a. tagged or discriminated unions) are a common tool in other languages; they allow a value to be one of a set of explicit cases, with easy and safe identification and data extraction. This article uses Redux state design to demonstrate common difficulties JS developers face when modeling a domain, shows how sum types mitigate those difficulties, and reviews a few libraries aiming to port sum types into the language.

Background: Redux State

Redux.js is “a predictable state container” inspired by the Elm Architecture. A developer can represent the canonical stateful data of their application in whatever form they wish, from a sophisticated Immutable.js Record to a straightforward POJO (Plain Old JavaScript Object). The state for an app which fetches and displays a list of adorable kittens might be as simple as:

const initialState = {
  kittens: [] // no kittens fetched yet 😿
}

Developers specify the state logic of a Redux-based app in a “reducer” function with the signature (oldState, action) -> newState.

function reducer (oldState, action) {
  if (action.type === 'GOT_KITTENS') {
    return { kittens: action.kittens } // replaces the kittens
  }
  return oldState // default, do nothing
}

Given an “action” object representing an application event, the reducer determines how to produce the new state.

newState = reducer(initialState, {
  type: 'GOT_KITTENS',
  kittens: ['Snuggles', 'Mittens']
})

console.log(newState) // { kittens: ['Snuggles', 'Mittens'] } 😺

User interface code (e.g. React components) can subsequently read the state, creating a list of kittens.

const currentState = reduxStore.getState()
const listItems = currentState.kittens.map(kitten =>
  <li>{ kitten.name }</li>
)

Aside: if you have never used JSX before, the above might appear unsettling. This domain-specific language compiles to vanilla JS:

const currentState = reduxStore.getState()
const listItems = currentState.kittens.map(kitten =>
  React.createElement('li', null, kitten.name)
)

Motivating Example: Tackling Complexity

Initially, this direct representation of state works as expected. The application starts with no kittens. Mapping over the empty array state.kittens produces no list items, and our UI shows nothing. Later, when kitten data is fetched and the state is updated, our list will pop into view (assuming the rest of our AJAX / Redux / React code is wired correctly).

In practice, however, the user may be confused upon being shown a blank page. We really ought to let them know that the kittens are on their way:

const kittens = reduxStore.getState().kittens

if (!kittens.length) { // kittens are loading?

  return <p>Calling the kittens!</p>

} else { // kittens received, show them

  return (<ul>{
    kittens.map(kitten =>
      <li>{ kitten.name }</li>
    )
  }</ul>)

}

Now when the user first loads the page, they see a lovely paragraph informing them of impending kittens (how exciting!). Later, the paragraph is replaced with a list of kitten names.

Signs of Trouble

One day, however, a user submits a help ticket. “When I visit the kittens page, it says they are loading forever.” What went wrong?

Well, on that day, the kittens data from the server was empty. That is, [] is a valid value representing the kittens in our database. We’ve hijacked the empty array to mean that the kittens are still loading, but that’s not necessarily true. We started with an initial state of 0 kittens:

{
  kittens: [] // intent: not yet fetched
}

And then ended up with a final state of 0 kittens:

{
  kittens: [] // intent: fetched (empty list from db)
}

These two states are indistinguishable, though we intended them to be distinct. Our mistake was conflating data (the array) with metadata (information regarding the array).

Falling Down the Rabbit Hole

As intrepid JS developers, we may next try distinguishing status based not on length, but on type. What if we use null to indicate unloaded kitties?

const initialState = {
  kittens: null
}

Unbidden, a wild error appears.

Error: cannot read property 'length' of null

In a way, we got lucky this time — the code failed noisily and immediately. The problem, of course, is that null values cannot have properties, so our old UI code checking kittens.length is broken. The fix isn’t especially difficult:

const kittens = reduxStore.getState().kittens

if (!kittens) { // kittens are loading

  return <p>Calling the kittens!</p>

} else if (!kittens.length) { // kittens are loaded but empty

  return <p>Sorry, no kittens available.</p>

} else { // kittens are loaded and can be shown

  return (<ul>{
    kittens.map(kitten =>
      <li>{ kitten.name }</li>
    )
  }</ul>)

}

That we have annotated the meaning of each case above should be considered a code smell; it reveals that our solution is not very semantic. Regardless, the unit tests pass, the app is deployed, and all seems well for a few days. Until…

When Zombies Attack

Error: cannot read property 'join' of null

Now what? Didn’t we solve this already? Ah, but this error is coming from another component:

const kittens = reduxStore.getState().kittens

return <p>'Known kittens include: ' + kittens.join(' & ')</p>

Oops. Someone forgot, or was never informed, that kittens might sometimes be null. The oversight escaped attention for a while because nobody ever ran this part of the application before the kittens were fetched. It was a trap waiting for the right edge case to come along.

Over time, multiple failures like this one are added and/or discovered. As long as developers think of state.kittens semantically as a collection of kittens, they keep trying to use it as an array — even if sometimes it isn’t one.

Out of the Frying Pan…

Fast-forward, and as product requirements have multiplied, so have the variety of ad-hoc solutions attempting to wrangle our state. The user stories now specify that kittens need to have four visible representations: unloaded, loading, fetched (with data), and failed (with an error). Across the app, similar needs are being dealt with for state.puppies and state.bunnies. Some leaves of the state tree use the strings unloaded and loading instead of the single valuenull, but that breaks some falsy checks: if (!state.bunnies) is now a mistake. The state.puppies leaf is being replaced with an Error object in the case of HTTP failure, forcing the verbose and unreliable check if (state.puppies instanceof Error) in the consuming code. Meanwhile, state.kittens is an object with .collection array and .isError boolean, forcing a different handling pattern — if (!state.kittens.isError) return state.kittens.collection[0]. Or maybe someone decided that some value might be false or null, and each stands for something different… good luck keeping them straight. With a deluge of ad-hoc reinvented cases to keep track of, developers are frequently forgetting to handle some of them, especially as the representations are inconsistent and inexpressive.

“Many languages have trouble expressing data with weird shapes. They give you a small set of built-in types, and you have to represent everything with them. So you often find yourself using null or booleans or strings to encode details in a way that is quite error prone.” — Elm Language, Union Types Documentation

JavaScript developers often consider tackling these burdens to be a normal part of writing code. There is a common assumption that one simply must be more careful not to make errors like those detailed above. Losing type safety is the tradeoff one accepts in return for JS’s flexibility, right? We see examples of this all the time, especially when searching for data:

• Array.prototype.indexOf returns -1 for “not found.” Not exactly clear.
• Worse, Array.prototype.find returns undefined for “not found;” how can we distinguish between “not found” and “found undefined?”
• Sequelize’s findById method returns a promise for null when database row is not found. It’s important to remember that might be the case, or else .then(user => user.name) could one day throw an error.

Programming paradigms that put the onus on human beings to be perfect inevitably result in confusion and mistakes. There must be a better way.

Sum Types, Defined in (Informal) Theory

The problems examined so far can be boiled down to two essential categories:

• No consistent and expressive way to represent separate data status cases. “Not found” ought to be clearly different from “found, with this data.”
• A high likelihood of error when consuming data that could be in one of several forms (i.e. polymorphic). Humans focus on the ordinary case and use variables assuming they will have certain properties or methods, forgetting that the actual value may sometimes be an exceptional case.

Sum types can improve both of the above. To understand sums, it helps to first examine scalar and product types.

Scalars

A scalar type contains atomic values: single items which cannot be decomposed into parts. Examples of scalars in JavaScript include built-in types like Boolean, Number, and Null. For instance, the number 42 in JavaScript is a single value inhabiting the Number scalar.

A scalar type has an intrinsic size (a.k.a. cardinality), which is just the number of values inhabiting that type.

• The Boolean type has a size of 2 as it contains two values, true & false.
• The Number type is of “infinite” size (if we ignore the limits of IEEE 754 ).
• The Null type has a size of 1 as it contains only one value, null.

Products

Product types are composite — they consist of multiple (possibly different, possibly identical) types, grouped together in an “and” fashion. For example, a type whose values consist of two Booleans (one Bool and one Bool) is a product. In JavaScript, we can represent custom product types using arrays as tuples, with position serving to distinguish each member type.

// FoodFacts are tuples of 2 bools: [isYummy, isHealthy]

const saladFacts   = [true, true]
const burgerFacts  = [true, false]
const vitaminFacts = [false, true]
const chalkFacts   = [false, false]

Why “product?” Because the size of this new composite FoodFacts type is determined by multiplying the sizes of its constituent types. We can easily see above that there are only four different values in the type, obtained by multiplying 2 options for the first Bool × 2 options for the second Bool.

Positional notation (e.g. chalkFacts[0]) is not very clear with respect to meaning. A more expressive way to represent multiple values grouped together in JS is with objects, which can label each member value. Technically the labels make objects record types rather than products, but we will overlook that in this article, in the interest of making it easier to write examples:

// Person  = { name: String, age: Number, employed: Boolean }

const mark = { name: 'Mark', age: 67, employed: false }
const jin  = { name: 'Jin',  age: 34, employed: true }
const ford = { name: 'Ford', age: 19, employed: true }
const sian = { name: 'Sian', age: 24, employed: false }
...

The size of the Person type, ignoring the labels, is Infinity × Infinity × 2. That is, the infinite number of Strings, times the infinite number of Numbers, times the two possible Booleans. Clearly, we will not be able to list every possible value in the Person type.

Sums

If scalar type size is an intrinsic number, and the size of a product type is the product of its constituent type sizes, you will probably not be surprised to hear that the size of a sum type is the sum of its constituent type sizes. Sum types are composite like product types, but in an “or” fashion; a single value in the type is only ever one of the constituent types, not a grouping of them all.

// FinitePrimitive = Boolean | Null | Undefined

const finitePrimitive1 = true
const finitePrimitive2 = false
const finitePrimitive3 = null
const finitePrimitive4 = undefined

The FinitePrimitive type we define above has a size of 2 + 1 + 1 = 4. That is, both of the Booleans, plus the single number of Nulls, plus the single number of Undefineds. We can easily list out all four values, which we have done above. Notice, a value in this type is only one of the constituent types.

As another example, consider a sum type composed of some larger types:

// InfinitePrimitive = String | Number | Symbol

const infinitePrimitive1 = 'hello'
const infinitePrimitive2 = 'goodbye'
const infinitePrimitive3 = 42
const infinitePrimitive4 = Symbol('hmm')
const infinitePrimitive5 = 314159
const infinitePrimitive6 = 'ok we get it, there are a lot of these'
...

The InfinitePrimitive type has a size of Infinity + Infinity + Infinity. It can be any one of the infinite strings, or one of the infinite numbers, or one of the infinite symbols.

A Sum of Products

Products and sums can consist of scalars, but we didn’t define them as needing to consist of scalar types — on the contrary, the constituent types may themselves be products and/or sums. Here is a (non-JavaScript) sum of both scalar and product types:

scalar type Ghost (contains one value, `ghost`)

product type Character { (contains 2 × 2 = 4 possible values)
  afraidOfNoGhosts: Boolean,
  ghostbuster: Boolean
}

sum type Entity = Ghost | Character

How many values does the Entity type have? Well, it’s 1 possible Ghost value + (2 × 2) possible Character values = 5 different Entity values. Let’s enumerate them:

entity1 = Ghost ghost
entity2 = Character { afraidOfNoGhosts: true, ghostbuster: true }
entity3 = Character { afraidOfNoGhosts: false, ghostbuster: true }
entity4 = Character { afraidOfNoGhosts: true, ghostbuster: false }
entity5 = Character { afraidOfNoGhosts: false, ghostbuster: false }

Tag, You’re It

We’ve almost finished defining sum types, but we’re missing a crucial characteristic which distinguishes them from the (very similar) union type. Suppose we define a sum type for name parts as being either a first name or a last name, where each is a string:

// NamePart can be a first name string OR a last name string
sum type NamePart = String | String

namePart1 = 'Wilson' // is this a first or last name?
namePart2 = 'Ashley' // is this a first of last name?

When we encounter a value in the wild, the fact that we know its type is String isn’t quite enough to know whether it was supposed to be from the first choice of NamePart strings, or the second choice.

For that, we need to somehow label the value — with a “tag.” The value of interest will not consist of just the string on its own, but also be accompanied by a symbolic identifier that allows the developer to know unambiguously which of the constituent types it belongs to:

namePart1 = <LastName 'Wilson'>
namePart2 = <FirstName 'Ashley'>

Ah, now we know exactly what roles 'Ashley' and 'Wilson' each play.

“This overall operation is called disjoint union. Basically, it can be summarized as ‘a union, but each element remembers what set it came from’.” — Waleed Khan, Union vs Sum Types

If you think about it, the combination of a tag and some data is itself a product, meaning we can reframe our example sum type as a sum of products, where every product includes a tag:

sum type NamePart = (FirstName & String) | (LastName & String)

Since each tag is only one value, it doesn’t affect the size of the sum type. NamePart now has a size of (1 × Infinity) + (1 × Infinity), equivalent to its earlier size of Infinity + Infinity. Tags are therefore unit types.

With tags, we can now discriminate between otherwise identical values of a given type; the tag is a minimal form of metadata. There is a preponderance of synonyms for the concept of sum types: discriminated unions, tagged unions, disjoint unions, choice types, variants, etc.

Sum types and product types, both being composite, are also known as algebraic data types.

Sum Types, Defined in Practice

We now have a loose theoretical understanding of what a sum type is, but what does one look like in actual code? For that we can turn to myriad typed languages which implement sum types as a native feature. Though they have been supported at least as far back as Algol68, using sum types as building blocks is especially important to many functional programming languages including Haskell, ReasonML, F#, Elm, and Rust.

Let’s compare and contrast an identical sum type across a couple different languages. The stock example, used in resources like Exploring ReasonML and Functional Languages and Learn You a Haskell for Great Good, is Shape:

• A Shape is a sum type, consisting of either a Circle or a Rectangle.
• Circle tags a product type, consisting of a Point (the center location) and a Float (the radius).
• Rectangle also tags a product type, consisting of a Point (one corner location) and another Point (the other corner).
• A Point is a product of two Floats (x and y coordinates).
• A Float is a scalar type; we will use IEEE 754 double precision floats as supplied by each language.

Shapes in ReasonML

ReasonML is an impure functional, eagerly evaluated, strongly typed language with type inference. It is essentially a JavaScript-like syntax for OCaml, which can compile to native code, JavaScript or even back to OCaml. It’s a great way for a JS native to start moving towards coding in a typed and more functional style. Sum types are referred to in ReasonML as variants.

“Behold, the crown jewel of Reason data structures!” —ReasonML Language, Variant Documentation

It should be emphasized that we are defining shape, Circle, Rectangle, and point here. The only built-in datatype we used was float. The sum type is shape, which can be a Circle or a Rectangle. We also construct a couple of example shape instances using each member tag.

Perhaps confusingly, Circle can be considered a tag, a constructor function, and a type; it has properties of all three. It is used to identify a value as being from one of the shape types; that makes it a tag. It is used to create values (like circ1) in the language; that makes it a constructor. And those values consist of two other types point and float grouped together; that is a product type. We can either use Circle as the name of that product of two types, or considering the implementation details, we could say that Circle is a value of the unit type within a product of three types. Whew!

On a side note, you might have noticed at the bottom of the code snippet that ReasonML ends up representing product types (like Point) as simple arrays in JavaScript, exactly as we showed in our earlier examination of products.

Shapes in OCaml

ReasonML can not only compile to JS but to OCaml, making it easy to see how we would define the same entities in that language.

OCaml’s type definition syntax, freed from the burden of seeming familiar to JavaScript developers, hews much closer to the purely theoretical concept of sum and product types — even to the point of using *.

The subsequent construction of instances is quite noisy with parens, on the other hand, making it clear why OCaml is sometimes called “LISP with types”:

Shapes in Haskell

Haskell is a purely functional, lazily evaluated, strongly typed language with type inference. Haskell has a heavy theoretical focus, though it is also used for practical purposes; it is very powerful yet exceptionally terse.

Again, the only built-in datatype we used was Float. Using :type in the GHCi REPL dutifully reports that yes, each example value has the type Shape. Had we added deriving (Show) to Shape, we could also log out circ1 in the REPL, which would display Circle (Point 2.0 3.0) 6.5. In short, our values “know” what types they come from and which tags they have.

Shapes in Rust

OK, one last example. Many of the languages that emphasize the use of sum types are dialects of, or were heavily influenced by, the ML family of languages: ReasonML / OCaml, Standard ML, F# etc. It’s difficult to find dramatically different syntaxes for sums, which makes evolutionary sense.

Rust is a nice example as it allows the developer to define product types (as structs) and sum types (as enums) using positional or record-style notation. This isn’t unique to Rust — Haskell can do the same, for instance — but Rust’s syntax demonstrates yet more names for types. Note also how constructing instances requires using the sum type as a namespace (Shape::Circle):

Don’t mistake the Rust enum for an ANSI C-style enum. The classic enumerated type is just a set of source code aliases for integers. True sum types allow the “arms” of the union to hold varied product types. Tagged unions can be implemented in C via a union of structs with an enum of tags and manual tag-checking code.

Shapes in JavaScript, Perhaps?

Having seen the same pattern encoded in a few languages, one might begin wondering how we can emulate it in JavaScript. The essence of sum types is that they consist of disjoint tagged types; the essence of tagging a type is to create a product of some identifier with the type. In JavaScript, a symbolic identifier with no other meaning should sound familiar; that’s a perfect use case for Symbol. And we already know some ways (arrays, objects) to loosely represent products in JavaScript; even if it cheats the theory a bit (again, records are technically not products), let’s use objects for clarity.

Oof. This concrete albeit unsophisticated first-pass attempt is not only much more laborious than the previous examples we saw, it is also incomplete. We do get some of the benefits correct:

• Expressive construction using namespaced and named factory functions
• Runtime type checks against constructor arguments
• Ability to discriminate unambiguously between values based on tag

On the other hand, we are missing some features:

• We have failed to truly create a sum type, as our rectangle and circle instances do not know they are shapes! This makes it difficult to specify that a function should receive any shape. We could manually write if (arg.tag !== CircleTag && arg.tag !== RectangleTag) in such a function, but that is brittle; what if we added a triangle case later?
• This code pattern is itself difficult, lengthy, and error-prone to replicate in a manual, ad-hoc fashion. It is also still abusable by human developers who may try to read the center of a rectangle or corner2 of a circle.
• Since JavaScript is dynamically typed, we only get failures at runtime. If a given function attempting to use a Shape is not immediately executed, we might not be aware that it contains a latent mistake.

Some of these points can be mitigated. Making values “remember” their types and tags together can be accomplished using a bit more code, and making it easier to define types in this fashion can be accomplished by abstracting the definition process to a function. Publish it as a library with a small API, and we’d be well on our way to capturing many of the benefits of sum types.

However, let’s not get ahead of ourselves. Before we see some practical real-world versions of JS sum types, we need to address the other half of the equation; not only constructing sum type instances, but consuming them.

The Payoff: Pattern Matching

By now you may be impatiently wondering what the point of all this theoretical messing about is. Recall the two problems we identified earlier: lack of a semantic way to separate cases, and human errors when using polymorphic data.

Sum types definitely solve the first problem. With the addition of a constructor tag, now every value comes with its own metadata identifier explaining just what the data’s case is.

How do we address the second problem? Ah, this is where sum types shine: pattern matching.

Pattern matching is a language-supported syntax for doing two things at once: identifying which case a value represents, and extracting the data from such a value. It lets the user of a sum type easily, declaratively, and safely consume values in the type. It prevents the user from mishandling the values by inverting control; the user no longer preemptively tries to inspect (potentially nonexistent) properties of an unknown type, but instead provides all possible type-handling cases in order to produce a result.

Let’s see pattern matching on our shapes via an example area function, with the signature Shape -> Float.

Pattern Matching in ReasonML

A ReasonML function can directly specify that it takes a shape. How do we use the shape? By matching all the tags using switch:

When we call area at the end, passing in a circle or a rectangle, the switch will match the passed-in shape to the correct case. Note that there is no JS-style fall-through — no more than one case will ever match. We should provide cases for every tag; if we omit one, the compiler will actually warn us we’ve forgotten something!

File "", line 14, characters 2-90: Warning 8: this pattern-matching is not exhaustive. Here is an example of a value that is not matched: Rectangle (_, _)

Exhaustiveness means that the function will be able to handle every value it could accept. An exhaustive function is also known as a total function. Non-total (i.e. partial) functions throw runtime errors if applied to a value they cannot handle.

Going in the other direction and attempting something nonsensical, like adding Point to our cases, stops the compiler outright:

Line 22, 6: This variant pattern is expected to have type shape The constructor Point does not belong to type shape

“But wait, there’s more!” Not only do we get to declaratively match to a specific case, but we can then destructure the data from that case using whatever variable names we want. The second argument of Circle is a float used for a radius; in our switch we pattern match to Circle and bind the second argument as the variable radius. We can also use _ to specify a value that we want to ignore, e.g. the center point for the circle; the area of a circle only depends on the radius after all.

Each case is a function from the matched pattern (with bindings from destructuring) to a result. In a very compact way, our code above states:

• if the passed-in shape is a circle,
• consisting of <nobody cares> and a radius,
• then return pi * radius²

Similar logic for the Rectangle case lets us destructure the x and y coordinates of each Point in the rectangle, and do the proper math.

Pattern matching in ReasonML can get more advanced; a common tool is a fallback case if no other pattern matches, which can be enumerated as _ => (your return value here). See here for other capabilities.

Pattern Matching in Haskell

Haskell also has a switch syntax, called case:

Calling area circ1 etc. in the REPL gives the expected results. Again, note that adding a nonsense case will cause the compiler to fail. Even more strict than ReasonML, omitting a case can also cause the compiler to fail, provided that you opt in to such behavior with the pragma {-# OPTIONS_GHC -Wincomplete-patterns -Werror #-}.

shape.hs:25:8: warning: [-Wincomplete-patterns]
    Pattern match(es) are non-exhaustive
    In a case alternative: Patterns not matched: (Circle _ _)

The popular LambdaCase language extension lets us to skip binding a name for the shape argument:

Haskell has other ways of defining area, including guards or simply writing each case as a separate equality:

Finally, Haskell allows a fallback case with otherwise.

Pattern Matching in Rust

Rust uses the somewhat more verb-oriented keyword match to perform pattern matching. Like when constructing, Rust also requires the developer to cite which sum type the tag comes from. When destructuring arguments from a record-style product type, it uses .. to ignore unused fields. Not shown, it can use _ to ignore positional fields and also to define a fallback case.

Again, omitting or exceeding the exhaustive cases causes the compiler to fail, informing the developer that something is wrong before any code is run.

error[E0004]: non-exhaustive patterns: `Rectangle(_, _)` not covered
  --> src/main.rs:10:18
   |
10 |   return match shape_arg {
   |                ^^^^^^^^^ pattern `Rectangle(_, _)` not covered

Sum Type Feature Wishlist

“Modelling data is important for a range of reasons. From performance to correctness to safety. Tagged unions give you a way of modelling choices that forces the correct handling of them, unlike predicate-based branching, such as the one used by if statements and other common control flow structures.” — Folktale Library, ADT Union Documentation

We’ve now seen sum types defined in theory, constructed in practice, and consumed in practice. If we were to list requirements for the ideal features of a language-supported sum type, they might read as follows.

Overall

• 📖 Feature a clean, expressive API
• 👮 Prevent naive direct inspection of data from a sum type value
• ✅ Enable easy type checking with an is or has function
• 💎 Luxury: enable extending sum types with methods
• 💎 Luxury: easy serialization / deserialization

Definition

• 🖋️ Define the name of the sum type
• 🗄️ Define the disjoint components of a sum type
• 📦 Allow each component to itself be another product, sum, or scalar type
• 🏷️ Label each component with a tag
• 👷 Have tags act as constructor functions, producing values from the type
• 📋 Enable tags to receive arguments, positionally and/or in record style
• 💎 Luxury: include integrated tools for working with product types

Matching

• 🔍 Enable matching a value to case by tag name
• 📜 Destructure component data during the match
• ➡️ Map destructured data to a return value for the match expression
• ❓ Allow for a catch-all fallback case in the match
• 🚨 Ideal: fail statically if possible (during compile-time) when omitting a required case or adding an incorrect case
• 🚨 Compromise: barring the above, fail immediately during runtime when omitting or adding a case
• 🚨 Worst: barring the above, fail eventually during runtime, upon discovering that a case cannot be matched

State of the Art: JavaScript Sum Type Libraries

Porting sum types into JS is hardly a new idea. A number of projects, some fairly prominent, have made efforts in this direction. Each implementation differs somewhat in terms of interface, capabilities, and guarantees. Below is a selection (far from exhaustive) of found examples, sorted by npm downloads per month:

Shapes in Folktale

Folktale is a general-purpose functional library with other tools besides adt/union. That inflates its numbers somewhat in the above table, but it also features one of the nicest (IMHO) discussions of tagged unions, so we’ll take a look at it. The embedded code below is editable, run it yourself.

Biggest knock: developers can directly access .radius off of a circle. This could tempt them to try and grab .radius from an unknown shape.

• Lack of an integrated product type is slightly awkward but not a big deal.
• Lack of mandatory declarative argument types is potentially dangerous. We can manually do the checking using static hasInstance, but that is a chore and relies on developers to dot their i’s and cross their t’s.
• Using vanilla JS destructuring with aliasing makes the pattern matching quite similar to the ideal.
• There is no exhaustiveness check or failure on extra cases yet, though it is an active issue (Folktale #138).
• There is no fallback case yet, though it is an active issue (Folktale #139).

Folktale has other features not shown, such as extension via shared derivations (presumably inspired by Haskell). Overall, it definitely gets us much of the way towards the ideal, but could stand to have better type safety.

Shapes in Daggy

Daggy is part of the wider Fantasy-Land ecosystem of functional JS specs and tools. The documentation is minimal, just two barely-explained example snippets; nothing we cannot understand, however.

• Biggest knock: developers can still directly access .radius off of a circle. This could tempt them to try and grab .radius from an unknown shape.
• Including an integrated product type is a nice touch.
• Only declaring field names is minimalist, but prevents us from even being able to perform type checking.
• Using positional fields is more powerful than forcing destructuring, as you could always use a single field and destructure it if you wanted.
• There is no exhaustiveness check or failure on extra cases, nor obvious plans to add either.
• There is no fallback case, nor obvious plans to add one.

Daggy allows extension through idiomatic prototypal inheritance — adding a method to Shape.prototype will allow Circles and Rectangles to delegate to that method.

Shapes in Union-Type

Conveniently, shape is one of the demonstrated examples from this library’s documentation (adapted slightly here).

• Biggest knock: developers can still directly access .radius off of a circle. This could tempt them to try and grab .radius from an unknown shape.
• We are back to the slightly awkward Point.Point as there is no distinct product type.
• We get all the benefits of declarative field names, plus validations, including declarative validations using both built-in and predefined types. Union-type does the type-checking part quite well.
• Relatively seamless handling of arrays or objects in defining and constructing values for the type, albeit only positional arguments during a match.
• There is no exhaustiveness check yet, though there is an issue for this (Union-Type #52).
• No failure on extra cases, nor obvious plans to add either.
• There is a fallback case, _, though it doesn’t help for shape per se.

Union-type is one of the most fully-featured examples at first glance. It includes prototypal inheritance, some fancy tools e.g. both instance and curried static case functions, variations like caseOn, recursive types, a fallback syntax and more.

Of note, one of the other libraries found (JAForbes/sum-type) was based on union-type, but with enhancements related to the sanctuary-def project.

So… Which Shall We Use?

This is just a small selection of libraries, and the comments above are not intended to make a final judgment as to viability, implementation details, and other concerns. Rather, the intent is to see how people have attempted to solve this issue to date, and consider how we would want such a library to work. We haven’t touched on potentially important features like serialization, for example. In short, given JavaScript’s dynamically-typed nature and variety of existing edge cases, any library attempting to implement sum types is likely to have some opinions baked in, and some issues left to iron out. For instance, none of the three libraries examined prevent developers from just directly attempting to grab a property from a sum type, even if not every member of the type has that property.

For the remainder of this article, I’ll be using union-type, mostly because it includes declarative type checks and a fallback case syntax.

Back to Redux

An age and a half ago, this article opened with an example JavaScript use case: a Redux state tree. If you recall, we were struggling to encode various states like “unloaded,” “loading,” “loaded with data,” and “failed with an error.” It should be clear that we could represent those states as a sum type.

If we had additional leaves in our state tree — state.bunnies, state.puppies, etc. — they would also be one of the four Leaf types. Our consuming code would use union-type’s case function to determine what UI to display.

It’s very likely that you’ve noticed something familiar about Redux’s action objects. Why… they’re just a tagged union themselves! An action can be one of a set number of objects, each with a type (tag), and each with potentially any number of other properties (making it a product type). In the reducer, we switch on each case based on type — #mindblown. Why not represent actions as a bona-fide sum type, then?

Unfortunately, Redux currently aggressively asserts that action objects be POJOs for now — no fancy library constructs. But the idea is sound, and in fact is precisely what inspired Redux in the first place. Redux is based on Elm, which includes tagged unions as a language feature. Recognizing that actions are members of a sum type, we have come full circle.

Redux Conclusion

So what have we gained? We are no longer reinventing the wheel, using primitive types to encode various mutually exclusive cases. And when we want to extract the data, we are reminded to handle every possible case. It’s not perfect — the type checks are limited, there are undoubtedly edge cases to consider, etc. And yet, the gain in both expressiveness, cleanliness, some degree of safety and more ought to be pretty appealing. Modeling your domain explicitly like this is a natural part of working with typed languages, and with all of JavaScript’s vaunted flexibility, it makes sense to adopt some of those benefits.

Case Studies (Pun Intended)

Redux state was a motivating example for this article, but sum types are so generally useful that it would be a shame not to go over some of their “greatest hits.” These are constructs so fundamental that they are often included in other language’s standard module. The following examples will be in pseudocode… see if you can implement them in a real language.

Maybe / Option

The Maybe type is either some data, or nothing at all.

sum type Maybe = Nothing | Just anything

firstOfList = list =>
    if (!list.length) return Nothing
    else return Just(list[0])

res = match firstOfList([])
    Nothing          => 'Sorry, there was nothing there.'
    Just (something) => 'Ah, found something:' + something

log(res) // 'Sorry, there was nothing there.'

No more -1, undefined, or null generating confusion. Methods like findIndex, find, and findById can now return a Maybe value; consuming code will then pattern match to decide what to do with the actual Nothing or Just something as the case may be. Try writing a safeDivide function which returns Nothing in case of divide by zero.

Cons List

This classic, recursive, closure-based, functional linked list is a workhorse of functional programming languages. Basically, a list is either the empty list Nil, or it is constructed from an element and a following list: Cons x xs.

sum type List = Nil | Cons anything List

myList = Cons(1, Cons(2, Cons(3, Nil)))

addList = someList =>
    match someList
        Nil         => 0
        Cons num xs => num + addList(xs)

log(addList(myList)) // 6

Binary Tree

Sum types excel at processing tree structures, which is part of why they are very handy in writing compilers.

sum type Tree = Leaf | anything Tree Tree

myTree = Tree(1,
    Tree(0, Leaf, Leaf), Tree(2,
         Tree(1.5, Leaf, Leaf), Tree(2.5, Leaf, Leaf)))

addTree = someTree =>
    match someTree
        Leaf => 0
        Tree num left right => num + addTree(left) + addTree(right)

addTree(myTree) // 7

Conclusion

When I started taking notes for this article, I told some friends that it would be “a quick blog post.” As it turns out, there was a lot more I wanted to say about sum types than I first realized. If you’ve made it this far, congratulations; I hope you enjoyed it and are interested to try using sum types in your next JavaScript project. The nature of the language may make the attempt imperfect, but an imperfect improvement is still an improvement. And with the possible future inclusion of native pattern matching, sum types in vanilla JS may become even more viable.

Resources

Here is a partial list of resources I found helpful in researching the subject, and can recommend for further reading.

Language Documentation

• ReasonML: Variant! [sic]. See also ReasonMLHub: Variant
• Haskell: Algebraic Data Type
• Elm : Union Types
• F#: Discriminated Unions
• Rust: Enums

JS Library Documentation

• Folktale: Union (particularly good doc-article hybrid)
• Union-Type
• Flow: Unions

Articles & Article Series

• Waleed Khan, Union vs Sum Types. Excellent article, really helped me understand the difference between these concepts.
• Gabriel Gonzalez, Sum Types. Examples in Haskell.
• Scott Wlaschin, F# for Fun and Profit: Designing with Types. Goes much further into how types can underpin the entire logic of an application. Also check out his talk on the same subject.
• Joel Burget, The Algebra (and Calculus!) of Algebraic Data Types. Goes more deeply into the theoretical and mathematical aspects of ADTs.

Wikipedia

• Tagged Union
• Algebraic Data Types

Chris Martin on Twitter

case analysis is the best got some sum type enumerating possible states handle them all compiler makes sure basically can't fuck up

Better JS Cases with Sum Types was originally published in Fullstack Academy on Medium, where people are continuing the conversation by highlighting and responding to this story.

Contrasting Code Executors & Translators via Concrete Examples

“I had a running compiler and nobody would touch it. …They carefully told me, computers could only do arithmetic; they could not do programs.” —Grace Hopper

Students learning high-level programming for the first time often struggle with differentiating the concepts of compilation and interpretation. This comes up frequently in the context of learning that Node.js and Chrome both share the same engine — V8— which “runs” JavaScript code. A common (though not entirely correct) beginner understanding often reads as: “interpreters run the code right away, but compilers output a binary which can be executed later.” This first-pass attempt at categorization suffices early on, but confusion may ensue when the student hears of a Just-in-Time (JIT) compiler: it’s a compiler which runs the code immediately? How is that different from an interpreter? And V8 used to feature two compilers, but now it has an interpreter too? Oh dear.

There are plenty of excellent resources in print and online for learning compilation theory. Their focus is usually on viable techniques for handling the translation of nontrivial languages: lexing a string into a token stream → parsing tokens into a concrete or abstract syntax tree, according to a formal grammar → generating or linearizing output from the tree.

Writing a working compiler using these techniques is satisfying and illuminating, but this article takes a different tack. We will define a trivial language “QuipScript,” which can be compiled with similarly trivial logic — no parse tree in sight. We will then compare & contrast running and/or translating a QS program via JS using:

• An interpreter
• A Just-in-Time (JIT) compiler
• An Ahead-of-Time (AOT) compiler
• An optimizing AOT compiler

With concrete and deliberately simple code examples, the intent is to make the differences between these approaches as intuitive as possible. Code for this article is located at https://github.com/glebec/int-jit-comp; let’s begin.

A Toy Language

QuipScript is a tiny language invented for this article that builds up, modifies, and prints a single stateful string value. Here’s an example QS program:

Bear in mind, a program is just a dead text file. It doesn’t necessarily do anything on its own — we need either a machine capable of using it directly, or else a system able to translate the text into something a machine can use.

This particular QS program is intended to first print “hello you”, followed by “hello world”. Take a look at the code and reason through how we get that result; it will be necessary to understand the next steps. When you’re ready, move on for a brief explanation.

Every QS program has a single implicit stateful string which begins empty. In this example, we first concatenate hello and then you into the string. Next, we print the current state of the string. Afterwards, we remove the last three letters, concat them, remove those four letters, concat world!, and remove the last letter. Our string state is now hello world — which we print.

Interpreter

Our QS code is certainly quite lovely (cough), but it’s useless without a way to make a computer execute it. As knowledgeable JavaScript programmers, however, it shouldn’t be too difficult to write a program which:

1. Reads in the text file as a string
2. Separates the string into lines
3. Identifies the intent of each line
4. Executes the specified logic in JavaScript, live

This is a QS interpreter, written in JS. Every statement in the QS code is read in and identified. When the JS code has determined the purpose of a QS statement, it executes the logic itself, in situ.

Running node interpreter.js produces the following output in our terminal:

hello you
hello world

Just as we wanted.

What if our source QS file had an error partway through — but after the first call to print? We might see output like this:

hello you
/Users/glebec/dev/compilers-article/interpreter.js:25
  default: throw Error('unexpected token: ' + line);
           ^

Error: unexpected token: ERROR I AM AN INVALID LINE OF GSCRIPT CODE
    at sourceCode.split.forEach.line (/Users/glebec/dev/compilers-article/interpreter.js:25:18)

Notice that hello you was printed before we reached the erroneous line of QS code. Interpreters merrily execute portions of source code as they go, meaning that they might perform some work before failing.

Interpreters generally have the following aims:

• Begin running immediately
• Produce results while analyzing source code
• Fail “close to” the error location

Caveat

Our QuipScript interpreter, written in JavaScript, can run statement by statement. This was easy since every QS statement conveniently maps 1-to-1 with an equivalent JS statement. A less contrived source language however might first require parsing into an intermediate language or tree representation in order to be correctly and/or efficiently interpreted.

Just-in-Time (JIT) Compiler

Our interpreter worked by figuring out the semantic intent of our QS code, and then realizing that intent itself as it read through the source. But that is not the only way we could have generated our final result.

As an alternative, we could compile (translate) our source QS code into target (or object) JS code — building up a program string. We won’t run each statement live, but rather wait until we have generated an entire JS program:

Provided there are no errors, the entire QS program will be converted into a JS program, before any source logic is actually executed. We finally use JavaScript’s (very dangerous) eval function to run our generated JS code.

Running the above code with node jit.js logs the following to our terminal:

hello you
hello world

FYI, here is the compiled program:

let string = '';
string += 'hello';
string += ' you';
console.log(string);
string = string.slice(0, -3);
string += 'them';
string = string.slice(0, -4);
string += 'world!';
string = string.slice(0, -1);
console.log(string);

It’s important to understand that the main output (hello you…) is not printed until the eval line runs. The generated JS program is also printed here just for demonstration purposes — in a real JIT we obviously would not embed the compiled code in the program output.

The main takeaway is that a compiler converts the input program into an equivalent target language program. A JIT compiler usually has these aims:

• To compile the entire source code to object code ASAP
• To subsequently run the object code, when it is possible to do so

Because our example compiler does not execute any source logic itself, if our source code includes a syntax error, the compiler will fail before any results are output (since the object code is never run):

/Users/glebec/dev/compilers-article/jit.js:31
  default: throw Error('unexpected token: ' + line);
           ^

Error: unexpected token: ERROR I AM AN INVALID LINE OF GSCRIPT CODE
    at sourceCode.split.forEach.line (/Users/glebec/dev/compilers-article/jit.js:31:18)

Notice, no hello you was printed this time.

Logically, it should make sense that an interpreter begins outputting results earlier than a JIT compiler. However, one would imagine that once the JIT has finished compiling, the generated JS program might theoretically run faster than the interpreter, which has to loop through and analyze each line. In our toy language example this might not be the case, but for more complex languages, interpreting the code is likely going to run slower than executing the compiled code.

Caveat

In reality, there are various more advanced flavors of JIT which combine the concepts of compilers & interpreters, compiling and running target code while parsing the source code. Some may also be capable of monitoring the running code and re-compiling source as necessary.

Many definitions of JIT outright require some or all of these additional capabilities. This article uses a more primitive distinction however, and separates JIT from AOT (see next section) based solely on whether the target code is immediately executed by the time it is done compiling.

Ahead-of-Time (AOT) Compiler

If we wanted to re-run our QS code, we could pass it through either the interpreter or JIT again. But then we’d be re-doing all the work from scratch. Wouldn’t it make sense to compile once, ahead of time, and simply save the generated JS program for future use?

This code is identical to the JIT above, except at the very end. Instead of immediately evaling our constructed JS program string, we save it to a file. So, running node compiler.js creates a new file (hello-world.js) for us to use in the future.

Subsequently entering node hello-world.js produces the expected output:

hello you
hello world

(Debug code truncated, as it is identical to the JIT example above.)

The advantage is clear: from now on, we don’t repeat work. We can run this output JS code as often as we want, and it will both start up immediately (like an interpreter) and run quickly (like a JIT). The downside is we had to produce the JS file before actually using it.

Optimizing AOT Compiler

If we are front-loading the work of compiling our source code (QS) to object code (JS), why are we allowing the JS to be so inefficient? Remember, our primitive compilers above generate the following JS program string/file:

let string = '';
string += 'hello';
string += ' you';
console.log(string);
string = string.slice(0, -3);
string += 'them';
string = string.slice(0, -4);
string += 'world!';
string = string.slice(0, -1);
console.log(string);

It isn’t hard to see that we could write a much simpler program in JS that accomplishes the same task. If only our compiler could intelligently do some of the work in advance, optimizing away inefficiencies:

This compiler is doing a bit of the work itself, finding ways of simplifying the program. When we use it via node opt-compiler.js, it saves the following code as hello-world.js:

console.log('hello you')
console.log('hello world')

Ah, much better! Running this code via node hello-world.js will give us exactly the results we want, with no superfluous string manipulation.

Doing the extra logic to generate this simplified code means an optimizing compiler might take longer than a JIT to compile, but the resultant JS code is even more efficient when actually run. In general, AOT compilers have the following aims:

• Analyze the code as deeply as possible, looking for optimizations.
• Generate output code which is fast (to run), small (on disk), and/or lean (in terms of memory use); priorities are not always mutually compatible.
• Save the generated code for future use, baking in improvements.

Optimizations

The “optimization” shown in this example was practically interpretation itself, but optimizations take many forms, including:

• Dead code elimination — if some source code could logically never be useful, remove it from the output. For example, if we did some more concatenation after the final print, that would be pointless work.
• Register allocation — the most used variables might be stored closer to the processor (e.g. in explicit registers), so they do not have to be retrieved from higher layers (e.g. RAM).
• Loop unrolling — if the number of passes of a loop can be analyzed ahead of time, a loop can be converted into a simple sequence of statements with no expensive “jump” commands or control flow statements.
• Inlining — if a variable is used just to name some constant data reused across the app, the compiler can simply replace the variable with the data in the output code (saving a memory lookup).

That’s just a small sampling; modern compilers analyzing well-designed languages can perform other sophisticated optimizations and static checks.

Case Study: V8

Google’s V8 Engine is the core of the Chrome browser and Node.js runtime environment. It is the code which reads your JS file and executes the logic within. So, how does V8 work? Is it an interpreter, a JIT, an AOT compiler, or something else entirely?

As it turns out, this is an area in which V8 is actively evolving. Let’s take a simplified look at both how it used to work and also the more recent pipeline.

Old Way: JIT + Optimizing Compiler

From 2010 to 2017, V8 used two compilers: a lazy JIT named Full-Codegen, and an optimizing compiler (either the earlier Crankshaft or later Turbofan). When V8 executed some JS, it would go through the following steps:

1. The JIT would analyze the JS and compile it to machine code for your particular platform. This takes a short period of time — longer than an interpreter takes to start, but generally short enough for humans not to care, often because the browser is still downloading other page elements. Interestingly, Full-Codegen would compile lazily — not converting everything at once, but targeting individual functions as needed.
2. The machine code would then be immediately executed by the runtime environment. Running machine code is much more efficient than interpreting JS. In the interest of getting your code to run ASAP, however, the JIT made little to no optimizations, so the machine code is not as fast as it could be.
3. As the machine code ran, V8 would profile it and identify hotspots — functions that are called frequently, potential bottlenecks to performance.
4. V8’s optimizing compiler (Crankshaft or Turbofan) would then kick in and re-translate those functions (from JS to machine code) in a more intensive way. This process is slower than JIT compilation, but the resulting machine code is faster to run.
5. V8 finally live-substituted in the newly-optimized machine code, straight into your running program. The performance of a script would improve while it was being used. Quite a neat trick!

New Way: Compiler + Interpreter + Optimizing Compiler

In 2017, V8 switched over to a new pipeline prominently featuring (drumroll)… an interpreter!? Yes, it may seem surprising, but the newest component, named Ignition, has some key advantages.

The trick is that Ignition does not interpret JavaScript per se, but rather a compiled intermediate language between JS and machine code — a platform-agnostic bytecode. Bytecode is essentially a formal pseudocode, taking the form of a low-level instruction set for a purely hypothetical processor. Because this deliberately generic language is similar to the assembly languages used in most modern computers, it is easy to take one last step and convert it into actual machine code for a given platform. One only needs to tweak the text to account for the local dialect, as it were.

So, V8’s newest pipeline actually features an (unnamed) JS-to-bytecode compiler, the Ignition bytecode interpreter, and the Turbofan optimizing compiler. When this system receives a JS file, the following occurs:

1. V8 compiles the source JS into generic bytecode, identical for every platform (not specific to a given OS or processor). The compiler uses a few “easy” optimizations in this step, so the bytecode is already fairly efficient.
2. Now, the Ignition interpreter begins executing the compiled bytecode. It may be slightly slower than running compiled machine code directly, but is still pretty fast.
3. Again, hotspots are identified and re-compiled using the Turbofan optimizing compiler. However, this time the optimization doesn’t start from scratch with JS, but proceeds from the already-generated bytecode.

At first glance, it is perhaps not obvious what advantages this new system confers. Can compiling to semi-optimized bytecode, and then interpreting that bytecode, be comparable to just doing a JIT compilation to machine code?

In fact, Ignition is quite close in execution speed to Full-Codegen, and the V8 developers reap the following rewards:

• Faster startup times, mostly because it is quicker to compile to bytecode than it is to fully compile the source JS to machine code.
• Significantly less memory use, because most of the source JS can now be “thrown away” after conversion to bytecode. This is especially good for mobile platforms, a major motivation behind Ignition’s development.
• Simpler integration with the Turbofan optimizing compiler, both speeding up the optimization step and making it easier to implement new JS language features (from ECMAScript) as they are published.

The moral of this story is that the archetypal view of interpreters resulting in slow runtime execution and compilers generating fast runtime execution can become muddied, or at least nuanced, when more complex hybrid solutions are considered.

Aside: WebAssembly

Having examined the differences between AOT and JIT compilation, you may very well ask: why don’t people compile their JS ahead of time, with an optimizing compiler? Wouldn’t that yield the best possible results?

The short answer is that it would be nice, and that’s how native apps work. A user downloads a precompiled binary for their OS and runs it directly. However, it’s not entirely feasible for the web, for the following reasons:

• Not all browsers are built on Chrome’s virtual machine. If websites had to serve up pre-compiled JS code to clients, they’d need multiple versions for different browsers and operating systems. Instead, all that browsers currently need is a single JS file. Different engines, e.g. SpiderMonkey (Firefox), Chakra (MS Edge), and JavaScriptCore (Safari) may interpret or JIT compile that JS, and they might compile it to bytecode or machine code. But in every case, JS is the lingua franca from which more vendor- and platform-specific results can be generated.
• For security reasons, it would be a bad idea to let browsers execute pre-compiled native code, which might read or write private data in the file system or perform other nefarious acts. By limiting browsers to JS, developers are restricted to only the APIs and capabilities that JS as a language affords (and which the browser grants).

However, the potential performance benefits of compiled code have not been overlooked. The W3C has designed a universal binary object code language for the internet, called WebAssembly. The promise of WASM is to provide nearly-native performance in a low-level, pre-compiled format, which is still secure and safe to be run by browsers (due to restricted capabilities).

Modern browsers are already supporting WASM, though JS is still needed e.g. to manipulate the DOM. Developers can write code in languages like C or Rust and compile to WASM modules which are loaded and invoked by JS. WASM therefore can greatly speed up certain kinds of calculations, working alongside JS to provide a more performant web.

Conclusion

This article is intended as a high-level introduction to lessen confusion among those encountering these concepts for the first time. The code snippets are not meant to demonstrate recommendable or effective techniques for use in real interpreters or compilers. However, it is my hope that these deliberately simple scripts might illuminate the core differences between these entities. Here is a summary of what we illustrated through examples:

It should also be evident that performing optimizations is not limited strictly to AOT compilers. A JIT can also perform some optimizations, so long as they do not slow down the compilation process too much. However, an AOT can afford to take as much time as it needs to generate the best possible result.

We also saw that more sophisticated pipelines may involve techniques such as lazy compilation, compilation to intermediate representations, dynamic re-compilation, and other hybrid approaches.

Additional Resources

If you want to learn more about this interesting and deep topic, here are some relevant resources:

• [GitHub] glebec/int-jit-comp (repository of runnable code examples from this article)
• [Amazon] Aho et al., Compilers: Principles, Techniques, and Tools (2nd ed.) (known as the “dragon book”, a classic text)
• [YouTube] McIlroy et al., Ignition: an Interpreter for V8 (with slides)
• [YouTube] Hinkelmann, JavaScript Engines — How Do They Even?
• [V8 Blog] Clifford, Launching Ignition and Turbofan
• [StackOverflow] Mittag, Understanding the Differences

Addendum

• [Medium] Hinkelmann, Understanding V8’s Bytecode

Fedor Indutny on Twitter

If you're curious about doing JIT assembly in JS, there is a very old blog post that may still work (or not): https://t.co/7YxmHXyTOE https://t.co/ZTeIY6g94a

Interpreters, JITS, & Compilers (Oh My!) was originally published in Fullstack Academy on Medium, where people are continuing the conversation by highlighting and responding to this story.

What Thunks Are, What They Solve, & Other Options

This article was born as a gist for React & Redux beginners, intended to demystify what thunks are and the motivation for using them.

Redux was created by Dan Abramov for a talk. It is a “state container” inspired by the unidirectional Flux data flow and functional Elm architecture. It provides a predictable approach to managing state that benefits from immutability, keeps business logic contained, acts as the single source of truth, and has a very small API.

The synchronous and pure flow of data through Redux’s components is well-defined with distinct, simple roles. Action creators create objects → objects are dispatched to the store → the store invokes reducers → reducers generate new state → listeners are notified of state updates.

However, Redux is not an application framework, and does not dictate how effects should be handled. For that, developers can adopt any preferred strategy through middleware.

I’d probably let action creator return a function. If it’s a function, it’s given dispatch and the state. — Dan Abramov, responding to issue #1 in Redux.

Redux-Thunk is arguably the most primitive such middleware. It is certainly the first that most people learn, having been written by Dan Abramov as part of Redux proper before being split out into a separate package. That original implementation is tiny enough to quote in its entirety:

Since then, the Redux-Thunk source code has only expanded to fourteen lines total. Despite this apparent simplicity, however, thunks still engender occasional confusion. If you find the concept fuzzy, fear not! We shall begin by answering a common question…

What are Thunks?

The precise definition of a “thunk” varies across contexts. Generally though, thunks are a functional programming technique used to delay computation. Instead of performing some work now, you produce a function body or unevaluated expression (the “thunk”) which can optionally be used to perform the work later. Compare:

Named functions help to highlight the thunk, but the distinction is made clearer using arrows. Notice how a thunk (the function returned from thunkedYell(…)) requires an extra invocation before the work is executed:

Here the potential work involves a side effect (logging), but thunks can also wrap calculations that might be slow, or even unending. In any case, other code can subsequently decide whether to actually run the thunk:

Aside: Laziness

Lazy languages like Haskell treat function arguments as thunks automatically, allowing for “infinite” computed-on-demand lists and clever compiler optimizations. Laziness is a powerful technique which can be implemented in JavaScript via various approaches and language features, including getters, proxies, and generators. For example, the Chalk library uses a getter to lazily build infinite property chains: chalk.dim.red.underline.bgBlue etc. On the mathematical front, there is a thunked version of the famous Y combinator, called (aptly enough) the Z combinator, which can be run in eager languages like JavaScript.

Laziness is a large topic deserving its own article. Rather than explore thunks and laziness in general, the remainder of this post will focus on redux-thunk.

Thunks in React & Redux

In React / Redux, thunks enable us to avoid directly causing side effects in our actions, action creators, or components. Instead, anything impure will be wrapped in a thunk. Later, that thunk will be invoked by middleware to actually cause the effect. By transferring our side effects to running at a single point of the Redux loop (at the middleware level), the rest of our app stays relatively pure. Pure functions and components are easier to reason about, test, maintain, extend, and reuse.

Fundamentals & Motivation

Redux store.dispatch expects to be applied to an "action" (object with type):

Because manually writing action objects in multiple places is a potential source of errors (what if we accidentally wrote userr instead of user?), we prefer "action creator" functions that always return a correctly-formatted action:

Problem

However, if we have to do some async, such as an AJAX call via the axios library, a simple action creator no longer works.

The problem is that asyncLogin no longer returns an action object. How could it? The payload data (user object) isn't available yet. Redux (specifically, dispatch) doesn't know how to handle promises – at least, not on its own.

First Thought: Call Async Directly

We could do a store.dispatch ourselves in the async handler:

This seems ok at first glance. However, it presents several cons.

Con A: Inconsistent API

Now our components sometimes call store.dispatch(syncActionCreator()), and sometimes call doSomeAsyncThing().

• In the latter case, it’s not obvious that we are dispatching to the store. We cannot identify Redux actions at a glance, making our app’s data flow more opaque.
• What if we later change an action-function from sync to async, or async to sync? We have to track down and modify the call site for that function, in every single component it is used. Yuck!

What we want is a way to still use store.dispatch(actionCreator()), even for async actions.

Con B: Impurity

The asyncLogin function isn't pure; it has a side effect (network call). Of course eventually we must make that call, and we'll see a solution soon. But side effects embedded in a component make that component harder to work with and reason about. For example, in unit testing, you may have to intercept or modify axios otherwise the component will make actual network calls.

Con C: Tight Coupling

The asyncLogin function is tightly coupled to a specific store in scope. That isn't reusable; what if we wanted to use this action creator with more than one Redux store, e.g. for server-side rendering? Or no real store at all, e.g. using a mock for testing?

Better: Thunks (Initial Attempt)

Enter thunks. Instead of making the network call now, you return a function which can be executed at will later.

We’re back to a single API style, and our action creator thunkedLogin is pure, or at least “purer”: when invoked, it returns a function, performing no immediate side effect.

“But wait,” an astute reader might object. “That action creator returns a function, which subsequently gets dispatched (last line). I thought Redux only understands action objects? Also, this is still tightly coupled!"

Correct. If this was our only change, the thunk would be passed into our Redux reducers, uselessly. Thunks are not magic, and insufficient on their own. Some additional code will need to actually invoke the thunk. That brings us to our next tool: whenever a value gets dispatched to the Redux store, it first passes through middleware.

Redux-Thunk Middleware

The redux-thunk middleware, once installed, does essentially the following:

• If a normal action object is dispatched, redux-thunk simply passes it along (e.g. into the reducer), as if redux-thunkdid not exist.
• If a function (e.g. a thunk) is dispatched, redux-thunk calls that function, passing in the store's dispatch and getState. It does not forward the thunk to the reducer.

Just what we needed! Now our action creators can return objects or functions. In the former case, everything works as normal. In the latter case, the function is intercepted and invoked.

When our example thunk is invoked by the middleware, it performs an asynchronous effect. When that async is complete,the callback or handler can dispatch a normal action to the store. Thunks therefore let us “escape” the normal Redux loop temporarily, with an async handler eventually re-entering the loop.

Dependency Injection

We have seen that thunks in Redux let us use a unified API and keep our action creators pure. However, our demonstration still used a specific store. The redux-thunk middleware gives us a way to solve that issue: dependency injection. DI is one technique for mitigating code coupling; instead of code knowing how to pull in a dependency (and therefore being tightly coupled to it), the dependency is provided to the code (and can therefore be easily swapped). This role reversal is an example of the more general concept of inversion of control.

Thunks generally take no arguments — they are latent computations, ready to be performed with no further input. However, redux-thunk bends that rule, and actually passes two arguments to the thunk: dispatch and getState. Our standard pattern for defining thunked action creators will therefore not need a scoped store:

How does this work? Where does this new dispatch argument come from?

The short answer is that the redux-thunk middleware has access to the store, and can therefore pass in the store's dispatch and getState when invoking the thunk. The middleware itself is responsible for injecting those dependencies into the thunk. The action creator module does not need to retrieve the store manually, so this action creator can be used for different stores or even a mocked dispatch.

getState

We did not show using getState in the thunk, as it is easy to abuse. In most Redux apps, it is more properly the responsibility of reducers (not actions) to use previous state to determine new state. There may be some cases in which reading the state inside a thunk is defensible, however, so be aware it is an option. Dan Abramov addresses using state in action creators:

Accessing Redux state in an action creator?

The few use cases where I think it’s acceptable is for checking cached data before you make a request, or for checking whether you are authenticated (in other words, doing a conditional dispatch) — Dan Abramov

withExtraArgument

“But wait, there’s more!” Not only does Redux-Thunk inject dispatch and getState, it can also inject any custom dependencies you want, using withExtraArgument. So if we wanted to inject axios, letting it be more easily mocked out for testing, we could.

At a certain point, one wonders where the dependency injection should stop. Is no code allowed to pull in dependencies? Is there a better way? DI and IoC are useful, but perhaps not ideal. Again, be aware of the option, but consider whether it is truly necessary for your application.

Why Thunk Middleware, and not Promise Middleware?

Promises are composable representations of asynchronous values, which have become native and widespread in JavaScript. The redux-promise and redux-promise-middleware packages enable dispatching promises or action objects containing promises. Both have some good capabilities and make handling async in Redux somewhat more convenient. However, neither addresses the issue of impurity. Promises are eager; they represent an async action that has already been initiated (in contrast to a Task, which is like a lazy Promise – user code is not actually executed unless you call run.)

Naive Promises Use

An initial attempt at using promises (without thunks) in Redux might appear as follows:

Look closely; this is essentially our “Call Async Directly” idea again. The promiseLogin function eventually dispatches an action from within a success handler. We are also dispatching the initial promise to the store, but what would any potential middleware do with that promise? At best, we'd want a hypothetical redux-promise-naive middleware to discard the promise so it doesn't end up in the reducer. That's doable, but overlooks some issues:

• Again, calling async code immediately makes our component / action-creator impure, which is difficult to work with and test.
• Our promiseLogin is still coupled to a specific store, reducing reusability.
• It can be tricky to distinguish a promise object from an action object. P/A+ promises have a painstaking [[promiseResolutionProcedure]] for duck-typing promises safely. The foolproof way to deal with this uncertainty is to coerce values using Promise.resolve, but doing so for every Redux action is a bit heavy-handed.

Smarter Promise Usage

The real redux-promise and redux-promise-middleware packages are smarter than our hypothetical redux-promise-naive. They allow dispatching promises or actions with promise payloads, and when the promise is fulfilled, the middleware will dispatch a normal action. For example:

Here, redux-promise-middleware will detect the explicitly declared payload.promise in a dispatched action. It prevents this action from going directly to the reducer, and automatically dispatches a separate 'LOGIN_PENDING' action instead. It then waits for the promise to settle, at which point it dispatches a 'LOGIN_FULFILLED' or 'LOGIN_REJECTED' action with the payload replaced by the promise's value or reason. That's a nice mix of actions we get for free, facilitating UI features like loading spinners or error notifications.

This middleware offers one improvement: promiseLogin no longer depends on a particular store. Rather, the middleware takes care of dispatching the final data to the store itself.

Unfortunately, redux-promise-middleware still hasn't contained the side effect; promiseLogin makes a network call immediately. This is arguably the Achilles' heel of promise-based redux middleware, and our components are back to being impure and needing some kind of hook or modification for testing purposes or reuse in other contexts.

Thunked Promises

As it turns out, nothing prevents us from using redux-promise-middleware alongside thunk-middleware. By delaying the creation of the promise, we gain both the laziness of thunks and the automatic action dispatching of redux-promise-middleware:

At this point, the simple core concept of thunks is beginning to be buried under complexities of debatable necessity. Do we really need two middleware libraries and to remember to use specific code patterns just to handle effects in Redux? We will examine a few alternatives shortly. Before then, there is one last note we ought to cover regarding promises and thunks.

Returning Promises from Thunks

When using redux-thunk, if a dispatched thunk returns a promise then dispatch will also return that same promise:

Once again, it is easy to abuse this pattern. One generally seeks to keep React components as pure as possible; adding async handlers back into them feels like a step backwards. It also makes our API inconsistent again.

However, there are a number of times and places where using a returned promise from a dispatch call can be nice. CassioZen presents a few in his ReactCasts #10: Redux Thunk Tricks video.

Alternatives to Thunks

Dan Abramov on Twitter

This is my karmic payback for redux-thunk

So, are thunks the One True Way to manage async / side effects in Redux applications? Certainly not. We already mentioned promise-based middleware. Thunks have at least one benefit over promises, but the below packages may be more convenient for certain use cases.

• Redux-Promise
• Redux-Promise-Middleware
• (2017–11–02 addendum: Redux-Pack also merits consideration)

Also, thunks are among the simplest of approaches. For any more complex asynchronicity, thunks can result in a lot of manual callback logic. More sophisticated, expressive, and composable solutions exist. The most established at this time, from most to least used, are:

• Redux-Saga, based on generator functions
• Redux-Observable, based on RxJS observables
• Redux-Loop, modeled after Elm’s effect system

Redux-Saga uses generators, a native feature all JavaScript developers ought to master. The remainder of Redux-Saga’s API is ad-hoc / unique, so while you may pick it up quickly, the knowledge is not as portable. It is especially nice for testing, however, as sagas return simple descriptions of desired effects, instead of functions which perform those effects.

In comparison, Redux-Observable is built on RxJS, a large library with a longer learning curve. However, RxJS is useful outside of Redux-Observable as a powerful and composable way to manage asynchronicity.

Redux-Loop is not as widespread, but follows Redux itself in being inspired by Elm. It is interesting for focusing not on action creators but rather reducers; this keep the state logic more central and contained.

These and other considerations make choosing between sagas, observables, loops, and any other options a matter of use case and preference, with no universal winner.

Conclusion

Ultimately, thunks are an effective solution for applications with simple async requirements. Understanding thunks is an approachable goal for those learning Redux the first time. Once you become comfortable with them, it’s a good idea to try alternatives.

Additional Resources

• Redux-Thunk Docs
• Dan Abramov: explaining thunks
• CassioZen: ReactCasts #10: Redux Thunk Tricks
• Dan Abramov: why you might or might not need thunks
• Dan Abramov: using state in action creators
• Mark Erikson: Idiomatic Redux: Thoughts on Thunks, Sagas, Abstraction, and Reusability

Thunks in Redux: The Basics was originally published in Fullstack Academy on Medium, where people are continuing the conversation by highlighting and responding to this story.