1. Introduction
2. One Expression to Rule Them All
3. Introduce Comments
4. Introduce Local Functions
5. Utility-Based Modularization
6. Domain-Based Modularization
7. Wrappers and Generics for Everything
8. Conclusion

Summary

Our code communicates implementation details (what the application does) best when we make it as concrete as it can possibly be.

Our code communicates our intent (why we made the implementation choices we did) best when we make it as abstract as it can possibly be.

When our code is perfectly concrete, intent in unclear, duplication is rampant, and change is hard because engineers attempting to read it experience detail-overload.

When our code is perfectly abstract, implementation details are unclear, meaningless partitions are rampant, and change is hard because engineers attempting to read it experience partition-overload.

When our code is perfectly concrete, the relationships between various parts of our code are unclear, because everything is in one enormous function, and so nothing exists to help distinguish the relationships.

When our code is perfectly abstract, the relationships between various parts of our code are unclear, because every single part is behind a partition. The partitions are therefore meaningless, and so nothing exists to help distinguish the relationships.

The relationships between various parts of our code are clear when we implement the best partitions for the given behavior, living within a given function, living within a given module, living within a given application, living within a given ecosystem. The definition of “best” will depend upon context, but it will always be somewhere between the two extremes of total concreteness and total abstraction, which are the least desirable configurations for any software project of significant size.

Takeaways for Elm

Component-based architecture forces the creation of partitions that are not tied to meaningful utilities or domains for a particular application. As a result, large component-based systems typically end up with some amount of abstraction that obscures clarity of detail without adding clarity of intent. One of the greatest aspects of programming in Elm as that the only abstractions we are beholden to are the signatures of the functions that we pass to Program.

However, I would love to see the Elm community discuss more openly which kinds of abstraction choices are good and which kinds of abstraction choices are bad, rather than urging individuals away from abstraction entirely. The time to start thinking about your application’s structure is not when it reaches six hundred lines. That time, in my experience, is always right now. Especially on a team, choosing any kind of arbitrary boundary, (for example, waiting until your data structures have such-and-such number of fields or until files become such-and-such lines long), is a recipe for engineers to proceed without thinking about structure. Then, the code will end up either overly concrete or overly abstract, and the team will have to deal with all the baggage that comes with that particular extreme.

1. Introduction
2. One Expression to Rule Them All
3. Introduce Comments
4. Introduce Local Functions
5. Utility-Based Modularization
6. Domain-Based Modularization
7. Wrappers and Generics for Everything
8. Conclusion

Code

Let’s suppose that I don’t care about how hard others have to work to find the details of my system’s behavior. Let’s say that clarifying my intent through complex structure is my highest goal. Let’s say I decide to go for extreme abstraction. I could do something like this:

Discussion

This code is now thoroughly more abstract than it needs to be. ColorMode might become worthwhile some day if we get to a point where we need four or five color values associated with each RunStatus, but this seems highly unlikely. Also, is there really a lot of value in abstracting the transition out of each part of oscillate? This has “solved” a duplication issue of only two elements that were unlikely to change (because they are tied to an external library). And making Model.RunStatus’s setters more generic has improved clarity of intent only marginally while introducing a new parameter to be parsed and understood by readers.

A little abstraction applied to concrete code initially yielded enormous returns in the form of duplication removal and readability. However, this code’s overly complex nature actively degrades its readability.

Analysis

Pros

• We have given names and partitions to all parts of the system in which even the minutest intent was present. Why we have the code that we do (our intent) is utterly clear.

Cons

• Learning simple concepts within the system requires learning about an enormous number of relationships between partitioned chunks of code. Some of these partitions have nothing to do with either utility or domain concerns.
• Discovering implementation details (the what) is time-consuming and involves mental contortions in order to hold the structure of the system in mind. This is necessary for even the simplest of operations.
• The relationships between various parts of the system, which initially started to improve as we added abstraction, have now degraded. This heavily abstract system is as unclear as the heavily concrete system in terms of how the parts of the system relate to one another.

Continue to Abstraction in Elm: Conclusion.

1. Introduction
2. One Expression to Rule Them All
3. Introduce Comments
4. Introduce Local Functions
5. Utility-Based Modularization
6. Domain-Based Modularization
7. Wrappers and Generics for Everything
8. Conclusion

Code

Did you think that we got rid of all the duplication when we introduced local functions? In fact, I intentionally kept a light touch on that first pass. There is still much more we could do.

In the previous section, we took the functions that were similar in their utility and grouped them into modules, hence the title of “utility-based modularization.” Utility, in this sense, means that we grouped functions together because they operated on similar data structures (RunStatus) or because they described a particular runtime behavior (ColorOscillation).

There is another kind partitioning that can be useful, and it can help us tease out some of the more subtle forms of duplication: abstraction based on domain.

Discussion

In the domain of returning a value from the update function, we need to wrap our model into a tuple with a Cmd. The And module abstracts this behavior and will attract other kinds of Cmds that our system needs to generate (one can imagine And.execute for simple port functionality or And.doSomeOtherBehavior for more complex situations.

In the domain of making changes to the model, we have four actions in this system, and those seem to belong to two categories. One pair of actions is concerned with making changes to runStatus, while the other is concerned with initiating animation effects. I partitioned these domains by introducing two new modules: Model.RunStatus and Model.Animation.

On the one hand, this kind of domain-driven partitioning can make the high levels of code extremely descriptive:

model
    |> Model.RunStatus.setToGeneratingTests
    |> Model.Animation.initiateColorOscillation
    |> And.doNothing

This reads wonderfully: “Take the model, set the run status to ‘generating tests’, then initiate a color oscillation animation, and then complete the update with no additional commands.”

The other side of this coin is that the specific behavior of this code is more obscure than ever. What do any of these lines of code really do? In order to go find that out, I’m now going to have to examine five other files, and additionally, I’ll have to keep the relationships of those five files to each other and to Main.elm in my head as I go.

Now, if my system is comprised of thirty message constructors (as Elm Test Runner happens to be at the time of this writing), then this kind of structure can be worth its weight, precisely for that supposed drawback. Think about how you go about understanding a large and complex system. When I first approach such a system, I don’t want to have to figure out how the entire application works in order to make a change to color animations! I just want to understand the color animation sub-system. Even though these pithy lines in main obscure the details, well-crafted partitions can make very large projects easier to comprehend, because they delineate sub-systems, allowing the engineer to learn those sub-systems one at a time.

This situation is much preferable to a large, low-abstraction software system, in which a sea of details must be waded and sifted for the bits relevant to the sub-system one wants to learn about. In my experience, large, low-abstraction projects are much more difficult to read, comprehend, and make changes to, than are large, high-abstraction projects, but only if the partitions delineate the sub-systems rather than slicing them to pieces.

Engineers coming to Elm from React, Angular, and Vue will do well to let go of DOM-based components as a default abstraction mechanism, and instead consider the various ways they could partition based on utility and domain in their Elm applications.

At the same time, the heavy level of abstraction described in this section must be used judiciously and is most useful when a system is so large that no normal human being is ever going to attempt to hold the whole thing in mind at a given moment.

Analysis

Pros

• If we’ve done a good job with our names, then the highest levels of our application (Main) read extraordinarily well.
• It is clearer than ever why the system was constructed the way it was. Intent is approaching crystal clarity.

Cons

• The details of our system, what it specifically does, have become hidden in an expanse of lower-level files.
• Multiple modules must be traversed to get from main to the specific behavior, and an engineer attempting to decipher the system must hold all of those module relationships in their head in order to figure out what’s going on.
• Interestingly, we have begun, once again, to obscure the relationships between various parts of the code. Domain-based partitions cause engineers to have to do extra work in order to form a mental picture of system cohesion.
• High-level readability has also come at the expense of ease-of-reasoning about the system as a whole. (This is mitigated somewhat if we have a very large system. Since it’s impossible for a human being to keep the whole system in mind anyway, having partitions that delineate the sub-systems makes it easier to learn whichever sub-system an engineer is interested in at a given time.)

Continue to Abstraction in Elm: Wrappers and Generics for Everything.

1. Introduction
2. One Expression to Rule Them All
3. Introduce Comments
4. Introduce Local Functions
5. Utility-Based Modularization
6. Domain-Based Modularization
7. Wrappers and Generics for Everything
8. Conclusion

Code

In the last section, we saw a lot of benefit from a little bit of abstraction. We introduced a handful of functions, and all of a sudden a bunch of relationships came into focus. Let’s take abstraction a step further and see what happens. Let’s introduce modules.

Discussion

In the previous section, we grouped together chunks of intermediate logic and put each one into its own function. In this section, we’ve grouped together functions and put them into separate modules. This act of putting like things together and separating them from unrelated parts of the system is called partitioning.

What do these new partitions get us?

Some would argue that this particular partitioning is superficial. And, for a chunk of code that was originally only forty lines, I would agree that this indeed seems to be more abstraction than is strictly necessary for a code sample of this size. However, most programs do not remain a mere forty lines for very long.

File-level partitioning abstractions (modules in Elm, classes in object-oriented languages) serve another important purpose that often goes unremarked, both in object-oriented and functional paradigms. Such abstractions act as “behavior attractors,” as Corey Haines describes in his book Understanding the Four Rules of Simple Design. The idea is that a well-considered partition acts as a magnet for new behaviors that “belong” conceptually inside its boundaries.

In the example above, say I need a new function that converts any given RunStatus to a display string. In the early examples, I would just throw that function in any old where, and maybe I wouldn’t even bother to wrap that behavior in a function. With my partitions in place, it is clear to anyone using this system that such behavior belongs in the RunStatus module.

Building a team attitude of always searching out the right partitions for the job makes everyone’s lives easier, because the placement and integration of new functionality will be both obvious and simple. In the extremely concrete systems we saw at the very beginning of this article, a new feature could be placed trivially, but correct integration of that behavior would be significantly more cumbersome.

The downside of this level of abstraction should be obvious. In our previous section, we introduced a level of indirection that caused us to go searching the file for other functions. In this section, we introduced a new level of indirection. Now, when seeing an unfamiliar symbol, we may additionally have to go searching our project for other files.

We can now see a pattern emerging. Every step we take in using structure to clarify why our code is the way it is, comes at the cost of another form of clarity: that of what the application specifically does.

Analysis

Pros

• Logical groups of functions, called modules in Elm, add increased clarity as to why our code is the way it is, and what its intended behavior is.
• Modules act as behavior attractors, making it easier to fit new features into the system (assuming the partitions are well-drawn).
• Relationships between the various parts of the code are clearer than ever, as we now have two tiers of partitions (functions and modules) that distinguish code as being more or less related to other parts of the system.

Cons

• The ease of discovery of what our code does has taken another hit. It is no longer possible to find out everything a program does by looking at a single file. When encountering an unknown symbol, the module it belongs to must be identified, found, opened, and scanned before we can know how it works.

Continue to Abstraction in Elm: Domain-Based Modularization.

1. Introduction
2. One Expression to Rule Them All
3. Introduce Comments
4. Introduce Local Functions
5. Utility-Based Modularization
6. Domain-Based Modularization
7. Wrappers and Generics for Everything
8. Conclusion

Code

Discussion

This code demonstrates a refactor in which many portions of the monolithic expression have been reorganized into separate functions. We are no longer living in a world of a single mega-function, but rather many small functions that each serve a discrete purpose.

What effects have these changes had?

The first and most obvious effect is the obliteration of the duplication present in both of the previous code examples. If I want to change the duration of the fade effect in the oscillation, I can now change it in one place and the entire system is affected. If I want to add a third animation transition to the oscillation and have it affect all oscillations in the system, there is one place to do that.

But the removal of duplication is not the only positive effect. This code communicates its intent at least as well as the comment method. But, unlike the comment method, the intent is baked into the structure of the code. In order to change that intent, an engineer will have to restructure the code in some new way, and they will be beholden to the compiler.

In addition, all these new functions we’ve added come with names, and those names provide additional contextual clues. If an engineer starts to make a change that violates the intent, a good name can give them pause to reconsider their change’s effect on the system.

And finally, we have created a new situation in our case statement that wasn’t possible before. Look how small it is compared to previous incarnations. And not only is it smaller, but examine how it reads: statusBarColorStyle = colorOscillation GeneratingTests model.statusBarColorStyle. If I am coming to this code for the first time, and I see this line, I know immediately, after reading only five discrete symbols, that this code likely starts a color oscillation effect on the status bar, based on the current status of GeneratingTests. This was not possible with our previous code. In order to arrive at this same conclusion in previous examples, we had to carefully read eleven full lines of code and probably also compare those eleven lines to other parts of our system just to be sure.

There is one downside, and it is non-trivial. In order to find out the details of how the colorOscillation works, the engineer has to go searching for its definition.

By introducing this abstraction, we’ve made our intent both clear and resilient to accidental change, but at the cost of making some of the details of our system harder to find.

Analysis

Pros

• The duplication has been eliminated.
• It is clearer why the code is the way it is, and it is difficult to change the code behavior independently of this communication.
• Relationships between different portions of code are clearer as well.

Cons

• We have slightly obscured what the code does. This is because we have introduced a small amount of indirection. Upon seeing colorOscillation for the first time, an engineer must go searching for its definition in order to discover how it works.

Continue to Abstraction in Elm: Utility-Based Modularization.

1. Introduction
2. One Expression to Rule Them All
3. Introduce Comments
4. Introduce Local Functions
5. Utility-Based Modularization
6. Domain-Based Modularization
7. Wrappers and Generics for Everything
8. Conclusion

Code

Discussion

It has been suggested that commenting is an appropriate mechanism for making code relationships clear. In some senses, commenting does the job. It can no longer be said that this code is completely ambiguous. The author’s intent is now clear, and the relationship between the run status and the animation is apparent as well.

However, I tend to call this level of abstraction (if that is an even appropriate term) “superficial abstraction.” There are two reasons for this. Comments do not address duplication, and they are highly prone to becoming stale.

Let’s suppose that, in the above code, I want to make a change to the duration of the color oscillation effect. With the current structure (which, to be clear, is the same as the previous example — utterly concrete), I must go searching for all instances of setting duration in a color oscillation and update them all individually. The rampant duplication is still a major problem. I can understand the code’s intent, but making changes remains cumbersome and error-prone.

The problems become more severe if I have a large team. Because comments have no impact on the compiler or the execution of the compiled software, it is possible to change the software’s behavior to do something other than what the comments say it does. When comments are the abstraction pattern of choice, engineers who change the code must remember to change both the source code and the associated comments. As the system grows more complex, comments may come to describe parts of the code that are not in their immediate vicinity (such as line 46 above). An engineer is unlikely to go searching through the code to find all the comments they have affected with a change to the code’s behavior, and the compiler will not be able to stop them. Comments such as these inevitably become lies, and when that happens, ambiguity will return in full force.

Analysis

Pros

• There is zero ambiguity in terms of what is being executed.
• Comments provide a simple way to clarify why the code is doing what it’s doing.
• Relationships between various portions of the code are now clear.

Cons

• The clarity of the why and the relationships are contingent upon engineers maintaining the cohesion of the code’s behavior and all associated comments with every change they make. As a system grows larger, comments will start to refer to parts of the code further and further from their own location, and dissolution becomes inevitable. At this point, the relationships and the why will become as muddled as having no abstraction at all.

Continue to Abstraction in Elm: Introduce Local Functions.

1. Introduction
2. One Expression to Rule Them All
3. Introduce Comments
4. Introduce Local Functions
5. Utility-Based Modularization
6. Domain-Based Modularization
7. Wrappers and Generics for Everything
8. Conclusion

Code

Discussion

Above are two branches of a case statement inside Elm Test Runner’s update function. However, I have adjusted the code to be as concrete as it can possibly be. I have removed all abstractions, save for those coming from an external library (elm-style-animation).

This code is very direct. It tells us exactly what it does, and it tells us all of what it does in one single chunk of one single file. However, at only forty lines, it is already, in some senses, “too much to grok.” runStatus is being set to Processing, and this is perhaps mostly clear. But what is the relationship between these particular status changes and the bar color being interrupted and looped in an animation? Why is the duration of each part of each loop 660? Can I change one duration number and not the others? And what are those color values for? Why are they all different?

And this is only forty lines. What if it were six hundred? If one were completely new to this code, it would take quite a while to work through all those lines and figure out why each and every detail does what it does. We would have to run the application, observe the behavior, and try to correlate it to various parts of the expression. All of this work would be made exponentially harder by the fact that, lacking any structure, it is unclear how any one part of our single mega-function relates to any other part.

To make matters even worse, this method of perfect concreteness is highly susceptible to duplication. In only forty lines, this code already manifests numerous duplicate structures. This is a side effect of our desire to have no abstraction — we must explicitly declare each function invocation with its specific parameters.

Analysis

Pros

• The code does exactly what it says it does. There is zero ambiguity in terms of what is being executed.

Cons

• This code lacks any information as to why any part of it is the way it is. Engineer intent is highly obscure.
• Relationships between various portions of the function are unclear. As the code grows linearly, this becomes exponentially more problematic.
• Duplication is inevitable.

Continue to Abstraction in Elm: Introduce Comments.

1. Introduction
2. One Expression to Rule Them All
3. Introduce Comments
4. Introduce Local Functions
5. Utility-Based Modularization
6. Domain-Based Modularization
7. Wrappers and Generics for Everything
8. Conclusion

Why Should We Care about Abstraction in Elm?

One of the largest challenges facing the Elm community today is creating an environment where React, Angular, and Vue engineers feel comfortable trying Elm, both as a language and a framework, while at the same time coercing them away from the “component-based design” abstraction pattern that React, Angular, and Vue have taught them. While necessary in order to work with those frameworks, it’s an unnecessary impediment when working with Elm.

The heavy push to discourage reliance on the component abstraction pattern is a good one, but it has also, in my opinion, led to some questionable conclusions. Notable examples would be Evan Czaplicki’s recommendation in “The Life of a File” that engineers not think about abstraction in Elm until a file is six-hundred lines long. I have also witnessed numerous instances in Elm Slack of community members discouraging abstraction outright, even making claims to the effect that “all abstraction obscures clarity.”

In this series, I will show that abstraction is not necessarily bad. Its effect on our code can vary enormously depending on the context and manner of its application. To demonstrate this, I will take two branches of an Elm case statement and take them from their most concrete state possible to their most abstract state possible, and I’ll talk about the effects that each of the changes has.

I will show that the phrase “all abstraction obscures clarity” is only half-true. Abstraction does decrease clarity of details (what our code actually does), but it also increases clarity of intent (why our code does what it does). Our goal when coding should be neither to maximize clarity of detail nor clarity of intent, for both strategies are fraught with significant drawbacks. We should instead optimize for clarity of relationships by using abstraction to draw meaningful partitions between various portions of our application. This can only be achieved if we correctly balance concreteness and abstraction when building our software.

Continue to Abstraction in Elm: One Expression to Rule Them All.

Elm Lens 0.3 is out today! In this release, I’ve introduced the settings panel. You can reach the settings panel for Elm Lens by opening Atom preferences, clicking on “Packages” in the sidebar, then clicking the “Settings” button on the Elm Lens package.

You can now toggle off individual pieces of the Elm Lens markup, if you decide you’d rather be without them. You can also adjust the opacity of the markup, if you’d rather it be less pronounced than the rest of your code.

Additionally, you can quickly toggle all of the display control checkboxes on and off with Ctrl-L.

There are a few other improvements in this release. Those of you who have been seeing weird square icons where the loading spinner and warning flag should be, should now see those icons correctly. Also, external references from unqualified imports should now be counted correctly.

But, let’s have a word about unqualified imports…

The Problem with Unqualified Imports

Way back in my early software engineering days, I became enamored of a feature in PHP that I thought was really cool. I discovered that you could configure the language to go looking for a class file auto-magically whenever you invoked its name with new. This allowed me to forego having to write a require statement for each class dependency at the top of my files. I could just new anything anywhere I wanted and PHP magic handled the rest.

As I gained more experience, I reached the inevitable conclusion: this feature was making my code worse. The convenience of not having to think about what I was newing and where made me prone to dependency clusters and less likely to think about my partitioning strategy.

Elm doesn’t have a lot of the troublesome baggage of PHP, but it does have dependencies. While there is no magic in Elm that will tell the compiler to import whole modules for you based on references, there is a syntax that allows you to leave out the specification for what you’re importing.

Instead of doing:

import ModuleA exposing (ModuleA, blarg, frangle)

It is possible to do this:

import ModuleA exposing (..)

This will import everything that’s exposed in ModuleA. From a dependency perspective, this is not great for your code. It means that if ModuleB imports ModuleA in this way, then it is potentially importing symbols that it doesn’t use. If I forget that ModuleB imports ModuleA in this way, I can add a ton of exposed functions to it, which will increase the coupling to ModuleB in a non-obvious way. It also creates a readability issue. In order to figure out which symbols in ModuleB come from ModuleA, I either have to comment out the import and see what breaks, or I have to go scan ModuleB. It would be better just to declare the imports explicitly.

I want Elm Lens to show you the right information about your dependencies, regardless of how your import statements are structured. However, you will get better performance with Elm Lens, and, I’m convinced, better code, if you declare your dependencies explicitly. I’ve found this principle to be true in any language with an import syntax (which is the vast majority of them). Explicit imports always end up paying back their cost in typing time and then some.

Elm Lens’s initial functionality introduced contextual markup for individual functions. Two out of three pieces of that metadata concerned references: places that the function in question got called. Elm Lens 0.1 showed you how many references each function had, but there was no way to see details of those references or quickly navigate to them.

Elm Lens 0.2 changes that. Now, when you click on the phrase “n internal references” or “n external references,” a detailed list of references displays in the new references panel at the bottom of the editor. Clicking on a reference in the panel will navigate the editor to the file and location of the reference.

You can get the new functionality by upgrading Elm Lens to the most recent version in Atom. If you haven’t tried Elm Lens yet, now’s a great time to check it out.