This going to be a little more technical. Basically I am documenting a problem I was trying to research for myself. It is a very basic intro do encrypting and decrypting a file using AWS KMS.

Encrypt

Let’s start with a single file we want to encrypt.

echo ‘Hello, Olivier’ > message.txt

Now we are ready to encrypt this message. I will assume that you already created a KMS key and have an ARN for said key. The command ( aws kms encrypt ) takes two mandatory arguments plaintext and key_id.

aws kms encrypt --plaintext file://message.txt --key-id arn:aws:kms:us-east-1:xxxxxxxx:key/xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxx

This will return something like this:

{
    "CiphertextBlob": "AQICAHgfMmH9u+060sH9MFBcUp3+puJthvFVikjor5isqUM3YQGJBIw/P9nY0K5scZh+OPGgAAAAbDBqBgkqhkiG9w0BBwagXTBbAgEAMFYGCSqGSIb3DQEHATAeBglghkgBZQMEAS4wEQQMh5PtPbfeWUobrdcbAgEQgCnmiRqRGyslnEQHIwpeN6mevKKWdpErlx2wK6v5IPqnXXWL5X0cEAasGw==",
    "KeyId": "arn:aws:kms:us-east-1:572028816325:key/xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxx"
}

We are really interested in the CiphertextBlob, so you can extract it like so and store it in a new file.

aws kms encrypt --plaintext file://message.txt --key-id arn:aws:kms:us-east-1:xxxxxxxx:key/xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxx --output text --query CiphertextBlob > message.encrypted.base64

If you know a little unix, you know that > message.encrypted.base4 means that the output of the previous command will be redirected to a new file in this case.

An astute reader may have noticed that I named the file with a .base64 extension, that is because by default the result of the encryption will be base64 encoded. For KMS to be able to decrypt this, the file must be decoded, as follows:

base64 --decode message.encrypted.base64 > message.encrypted

You could have done all of the above in one go, using the following command.

aws kms encrypt --plaintext file://message.txt --key-id arn:aws:kms:us-east-1:xxxxxxxx:key/xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxx --output text --query CiphertextBlob | base64 --decode > message.encrypted

If you check your current folder now you will find the following files:

• message.txt — your original file
• message.encrypted.base64 — the encrypted version encoded in base64
• message.encrypted — the encrypted version

Decrypt

To decrypt, we basically have to do the reverse, sort of.

Step one is to decrypt. The command ( aws kms decrypt ) takes two mandatory arguments ciphertext-blob.

aws kms decrypt --ciphertext-blob fileb://message.encrypted

Note the b in fileb://. Also, no need to specify the key. As long as you are in your account AWS knows which key to use to decode your message.

The output of the command above will be something like:

{
    "KeyId": "arn:aws:kms:us-east-1:xxxxxxx:key/xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxx",
    "Plaintext": "MjgxMSXNb2JyaXNvbgo="
}

Of course, we are only interested in the plain text. So let’s modify the command above to extract what you need. And let’s store that in its own file, so we can see what we are doing.

aws kms decrypt --ciphertext-blob fileb://message.encrypted --output text --query Plaintext > message.decrypted.base64

If view the content of your output file cat message.decrypted.base64, you will see that it still not the original text that was in our file.

One more thing to do … base64 decoding.

base64 --decode message.decrypted.base64 > message.decrypted

cat message.decrypted
Hello, Olivier

Alleluia. We encrypted and decrypted a message.

Going further

What is the use in encrypting Hello, Olivier? We recently had to store a private key in AWS. We used this technique to encrypt the private key, prior to uploading it.

Can you think of any other use cases?

Like many technology organizations, Customer Care Technology, my team at The New York Times, uses scrum as our main process. To plan our sprints, we would go through the backlog, pick stories and assign a story to each of our six engineers, who would then work hard to get their story done. Because of unforeseen roadblocks, the stories would often get pushed to the next sprint or we would work together on a story to finish it before the end of the sprint. The problem with this process was that by the time we realized we needed to work together, it was often too late in the sprint.

Recently, we started working together, or swarming, for our entire sprints and only taking on a small number of stories at a time. Swarming is not a new concept; Many agile teams often swarm on a story to finish it before moving on to a new story. We took the definition of swarming and made it or own: We don’t just swarm to finish what has already been started, we swarm as soon as we start something new. And we’ve become more productive and collaborative as a result.

How we got here

A few months ago, the head of our department had a meeting with our team to encourage us to try a new way of working. As an engineering organization, it is important to focus on the right metrics, so we often focus on utilization or efficiency. However the more important metric is throughput: How much value can we deliver to the business and how quickly can we achieve this?

Translate this to scrum. Focusing on reducing the amount of inventory (work in progress) we have at any given time means we can execute faster. Working on fewer stories at one time means we can finish more, faster.

To understand how this works in practice, imagine this sprint scenario: We start with one story (let’s call this Story One), but Story One gets blocked by an external dependency, so we move on to the next story (we’ll call this Story Two). We have more work in progress. Because we started on Story Two late in the sprint, it is not done by the end of the sprint. During the next sprint planning we identify three new high priority stories (Stories Three, Four and Five). Now there is even more work in progress.

At this point, the dependency that blocked Story One is resolved, so we switch gears to work on Story One again. But when our product manager asks us when we can get to Story Six, we have to finish stories one through five first. Increased work in progress hinders throughput. It takes longer and longer to get anything done.

The head of our department suggested that instead of assigning one story to each engineer, multiple engineers should work simultaneously on one story. Thus we would have less inventory, less waste and increased throughput.

At our first sprint planning meeting where we were going to swarm for an entire sprint, we divided the team into subteams and took on only two or three stories. When my subteam met for the first time, we looked at each other and scratched our heads; Trying to figure out how to divide the work was awkward.

The idea behind swarming is to divide a story into smaller tasks that can be done in parallel, but sometimes it can be hard to divide the work because the tasks can end up being very interdependent and even overlap.

Different stories will require different solutions. Here are some of the ways we tackled this.

Techniques for Swarming

Divide by functionality

This may be the easiest and most logical one. If the story is, “Display a list of transactions, and for each transaction, there should be a ‘delete’ button to delete that transaction,” one engineer could work on displaying the list, while another could work on the ‘delete’ aspect.

Another example, if the story is, “build new functionality that should only show up under certain circumstances,” one engineer could start working on the conditional display of the feature, while another engineer works on the feature itself. It is probably the most natural way of dividing up work, but everyone would still need to work closely together to coordinate the touchpoints between the two tasks.

Divide by front-end and back-end work

This is pretty obvious as well and is an easy way to divide work. Very often an engineer can start working on the UI, while another can work on the back-end. Again, working closely together is key. Agree on the contract that will bind the front-end and back-end first before you start working.

Pair program

Two engineers work together on one machine: one person is the driver and one person is the navigator. The driver writes the code and the navigator sets the direction, observes and reviews. The benefits of pair programming are well documented: fewer bugs, shared knowledge, more efficient problem solving. As part of our effort to swarm, we have done our share of pair programming as well. Especially for more thorny issues or complicated features, it has proven quite valuable.

Test-driven development

If you practice true test driven development, you’re supposed to write the test first, see it fail and subsequently write the feature. In our version of swarming, one engineer starts writing component tests and/or integration tests, while the other one works on the underlying functionality.

Integrating QA in the development process

We have gotten much better about integrating QA in our development process. We used to loop QA in after our code was pushed to the master branch, which was clunky. Now our QA engineers will identify and create a test plan as soon as we begin working on a new story. As initial development nears completion, an engineer and a QA engineer will run through test scenarios together on our local machines.

The QA engineer may notice something that we had not thought of, or something that was missed in development, and at that point it is easy to remedy and fix. When the QA engineer likes what they see, we merge the branch into master and QA can verify it again on the dev server.

The way we are doing QA now has been a huge revelation to me and has had a major impact on our team. Because QA is more involved with the coding earlier on in the process, we have been able to develop features much faster. Bugs and unforeseen features are discovered earlier in the process and can be handled immediately.

Intense planning and coordination

In order to make sprint-length swarming work, you need planning and more planning. We typically have a story lead who meets with the members of the team and together, they break the story into subtasks in the way that makes the most sense. It is a collective planning effort. At that point we also agree on the contracts between the various subtasks. We also rely on the story lead to perform any needed coordination as we move along with the development.

It is often believed that scrum involves less (project) planning, but we all know that this is not true. When you swarm, you need even more planning.

Takeaways

After “swarming” for a few sprints, we started to get the hang of it. But ultimately, we are increasing throughput, which means we are completing features faster and delivering more.

There are some other benefits as well.

Few problems in this world don’t benefit from increased communication and software development is no exception. We talk about how to divide up the work, and we stay in close contact while we’re working because our projects often overlap. This means we can identify issues or roadblocks early in the sprint and correct accordingly.

I would also argue that increased communication leads to better software design decisions. We often have a healthy debate about the best way to tackle problems, and we probably wouldn’t have these conversations if we worked independently

By working closely together and discussing new features, we inevitably learn from each other: we learn about our specific domains, about the application we’re working on and general software building best practices.

All in all, this has been a very positive experience for our team and I encourage everyone to try it. If you have any questions, feel free to reach out.

Scrum, Swarm, Sprint — How to Take the Agile Process and Make it Your Own was originally published in NYT Open on Medium, where people are continuing the conversation by highlighting and responding to this story.

Frontend

This is an easy one. React had a meteoric rise in 2017, effectively overtaking Angular as the most popular front-end framework. Will there beanother framework that overtakes React? Looks like Vue.js is the new kid on the block. My prediction: React will continue to be the lingua franca of the front-end. Many developers love React. I just don’t see why developers would leave React in droves.

Backend

A few years ago Node.js made a big splash, but is now plateauing a bit. Ruby on Rails is no longer the coolest kid. Corporate still does Java. Php is still around. Python is gaining popularity, because of machine learning and because it’s fun. And by extension we’re seeing Django getting a new boost. But 2018 is the year of .NET’s explosion. Microsoft open sourced its framework a few years ago. .NET core 2.0 came out last year, and it is cross-platform. Developers are going to realize that it is no longer M$FT but that Satya Nadella really turned the company around. C# is a powerful language, there is a huge eco-system, you don’t have to use Visual Studio, you can do back-end, web, mobile apps, libraries, you can run it on linux or on your mac. Watch this space.

Infrastructure

Will 2018 be the year that Serverless becomes all the talk? Maybe, … but I don’t see an immediate mass adoption of cloud functions. My advice to developers: be prepared to learn DevOps, deployment pipelines and container orchestration services. Docker containers have been around for a while, Kubernetes was first released in 2014. It’s the way of the future. Amazon and Netflix were maybe pioneers in the space of microservices and the ‘you build it, you run it’ model. But now many companies — big and small — have caught on. It’s ready for the masses. See many legacy system being converted to containers. See hobbyists embrace it. One caveat, it’s a steep learning curve and there is a lot to learn. My prediction: someone will develop a tool or app to make deployment and container orchestration much more seamless and it will take off.

Tying it all together

My last prediction is pretty easy too, encompassing front-end and back-end. Rest has been around for too long without major innovation and is ripe for the taking. Enter GraphQL. I predict a majority of job listings for software engineers and web developers will list GraphQL as required. And as usual in the recruiting world, they will want to see 3–5 years of experience. (Facebook released GraphQL in 2015 :-)

2018 Web Development Predictions was originally published in HackerNoon.com on Medium, where people are continuing the conversation by highlighting and responding to this story.

As the title states, JavaScript closures have always been a bit of a mystery to me. I have read multiple articles, I have used closures in my work, sometimes I even used a closure without realizing I was using a closure.

Recently I went to a talk where someone really explained it in a way it finally clicked for me. I’ll try to take this approach to explain closures in this article. Let me give credit to the great folks at CodeSmith and their JavaScript The Hard Parts series.

Before we start

Some concepts are important to grok before you can grok closures. One of them is the execution context.

This article has a very good primer on Execution Context. To quote the article:

When code is run in JavaScript, the environment in which it is executed is very important, and is evaluated as 1 of the following:

Global code — The default environment where your code is executed for the first time.

Function code — Whenever the flow of execution enters a function body.

(…)

(…), let’s think of the term execution context as the environment / scope the current code is being evaluated in.

In other words, as we start the program, we start in the global execution context. Some variables are declared within the global execution context. We call these global variables. When the program calls a function, what happens? A few steps:

1. JavaScript creates a new execution context, a local execution context
2. That local execution context will have its own set of variables, these variables will be local to that execution context.
3. The new execution context is thrown onto the execution stack. Think of the execution stack as a mechanism to keep track of where the program is in its execution

When does the function end? When it encounters a return statement or it encounters a closing bracket }. When a function ends, the following happens:

1. The local execution contexts pops off the execution stack
2. The functions sends the return value back to the calling context. The calling context is the execution context that called this function, it could be the global execution context or another local execution context. It is up to the calling execution context to deal with the return value at that point. The returned value could be an object, an array, a function, a boolean, anything really. If the function has no return statement, undefined is returned.
3. The local execution context is destroyed. This is important. Destroyed. All the variables that were declared within the local execution context are erased. They are no longer available. That’s why they’re called local variables.

A very basic example

Before we get to closures, let’s take a look at the following piece of code. It seems very straightforward, anybody reading this article probably knows exactly what it does.

1: let a = 3
2: function addTwo(x) {
3:   let ret = x + 2
4:   return ret
5: }
6: let b = addTwo(a)
7: console.log(b)

In order to understand how the JavaScript engine really works, let’s break this down in great detail.

1. On line 1 we declare a new variable a in the global execution context and assign it the number 3.
2. Next it gets tricky. Lines 2 through 5 are really together. What happens here? We declare a new variable named addTwo in the global execution context. And what do we assign to it? A function definition. Whatever is between the two brackets { } is assigned to addTwo. The code inside the function is not evaluated, not executed, just stored into a variable for future use.
3. So now we’re at line 6. It looks simple, but there is much to unpack here. First we declare a new variable in the global execution context and label it b. As soon as a variable is declared it has the value of undefined.
4. Next, still on line 6, we see an assignment operator. We are getting ready to assign a new value to the variable b. Next we see a function being called. When you see a variable followed by round brackets (…), that’s the signal that a function is being called. Flash forward, every function returns something (either a value, an object or undefined). Whatever is returned from the function will be assigned to variable b.
5. But first we need to call the function labeled addTwo. JavaScript will go and look in its global execution context memory for a variable named addTwo. Oh, it found one, it was defined in step 2 (or lines 2–5). And lo and behold variable addTwo contains a function definition. Note that the variable a is passed as an argument to the function. JavaScript searches for a variable a in its global execution context memory, finds it, finds that its value is 3 and passes the number 3 as an argument to the function. Ready to execute the function.
6. Now the execution context will switch. A new local execution context is created, let’s name it the ‘addTwo execution context’. The execution context is pushed onto the call stack. What is the first thing we do in the local execution context?
7. You may be tempted to say, “A new variable ret is declared in the local execution context”. That is not the answer. The correct answer is, we need to look at the parameters of the function first. A new variable x is declared in the local execution context. And since the value 3 was passed as an argument, the variable x is assigned the number 3.
8. The next step is: A new variable ret is declared in the local execution context. Its value is set to undefined. (line 3)
9. Still line 3, an addition needs to be performed. First we need the value of x. JavaScript will look for a variable x. It will look in the local execution context first. And it found one, the value is 3. And the second operand is the number2. The result of the addition (5) is assigned to the variable ret.
10. Line 4. We return the content of the variable ret. Another lookup in the local execution context. ret contains the value 5. The function returns the number 5. And the function ends.
11. Lines 4–5. The function ends. The local execution context is destroyed. The variables x and ret are wiped out. They no longer exist. The context is popped of the call stack and the return value is returned to the calling context. In this case the calling context is the global execution context, because the function addTwo was called from the global execution context.
12. Now we pick up where we left off in step 4. The returned value (number 5) gets assigned to the variable b. We are still at line 6 of the little program.
13. I am not going into detail, but in line 7, the content of variable b gets printed in the console. In our example the number 5.

That was a very long winded explanation for a very simple program, and we haven’t even touched upon closures yet. We will get there I promise. But first we need to take another detour or two.

Lexical scope.

We need to understand some aspects of lexical scope. Take a look at the following example.

1: let val1 = 2
2: function multiplyThis(n) {
3:   let ret = n * val1
4:   return ret
5: }
6: let multiplied = multiplyThis(6)
7: console.log('example of scope:', multiplied)

The idea here is that we have variables in the local execution context and variables in the global execution context. One intricacy of JavaScript is how it looks for variables. If it can’t find a variable in its local execution context, it will look for it in its calling context. And if not found there in its calling context. Repeatedly, until it is looking in the global execution context. (And if it does not find it there, it’s undefined). Follow along with the example above, it will clarify it. If you understand how scope works, you can skip this.

1. Declare a new variable val1 in the global execution context and assign it the number 2.
2. Lines 2–5. Declare a new variable multiplyThis and assign it a function definition.
3. Line 6. Declare a new variable multiplied in the global execution context.
4. Retrieve the variable multiplyThis from the global execution context memory and execute it as a function. Pass the number 6 as argument.
5. New function call = new execution context. Create a new local execution context.
6. In the local execution context, declare a variable n and assign it the number 6.
7. Line 3. In the local execution context, declare a variable ret.
8. Line 3 (continued). Perform an multiplication with two operands; the content of the variables n and val1. Look up the variable n in the local execution context. We declared it in step 6. Its content is the number 6. Look up the variable val1 in the local execution context. The local execution context does not have a variable labeled val1. Let’s check the calling context. The calling context is the global execution context. Let’s look for val1 in the global execution context. Oh yes, it’s there. It was defined in step 1. The value is the number 2.
9. Line 3 (continued). Multiply the two operands and assign it to the ret variable. 6 * 2 = 12. ret is now 12.
10. Return the ret variable. The local execution context is destroyed, along with its variables ret and n. The variable val1 is not destroyed, as it was part of the global execution context.
11. Back to line 6. In the calling context, the number 12 is assigned to the multiplied variable.
12. Finally on line 7, we show the value of the multiplied variable in the console.

So in this example, we need to remember that a function has access to variables that are defined in its calling context. The formal name of this phenomenon is the lexical scope.

A function that returns a function

In the first example the function addTwo returns a number. Remember from earlier that a function can return anything. Let’s look at an example of a function that returns a function, as this is essential to understand closures. Here is the example that we are going to analyze.

 1: let val = 7
 2: function createAdder() {
 3:   function addNumbers(a, b) {
 4:     let ret = a + b
 5:     return ret
 6:   }
 7:   return addNumbers
 8: }
 9: let adder = createAdder()
10: let sum = adder(val, 8)
11: console.log('example of function returning a function: ', sum)

Let’s go back to the step-by-step breakdown.

1. Line 1. We declare a variable val in the global execution context and assign the number 7 to that variable.
2. Lines 2–8. We declare a variable named createAdder in the global execution context and we assign a function definition to it. Lines 3 to 7 describe said function definition. As before, at this point, we are not jumping into that function. We just store the function definition into that variable (createAdder).
3. Line 9. We declare a new variable, named adder, in the global execution context. Temporarily, undefined is assigned to adder.
4. Still line 9. We see the brackets (); we need to execute or call a function. Let’s query the global execution context’s memory and look for a variable named createAdder. It was created in step 2. Ok, let’s call it.
5. Calling a function. Now we’re at line 2. A new local execution context is created. We can create local variables in the new execution context. The engine adds the new context to the call stack. The function has no arguments, let’s jump right into the body of it.
6. Still lines 3–6. We have a new function declaration. We create a variable addNumbers in the local execution context. This important. addNumbers exists only in the local execution context. We store a function definition in the local variable named addNumbers.
7. Now we’re at line 7. We return the content of the variable addNumbers. The engine looks for a variable named addNumbers and finds it. It’s a function definition. Fine, a function can return anything, including a function definition. So we return the definition of addNumbers. Anything between the brackets on lines 4 and 5 makes up the function definition. We also remove the local execution context from the call stack.
8. Upon return, the local execution context is destroyed. The addNumbers variable is no more. The function definition still exists though, it is returned from the function and it is assigned to the variable adder; that is the variable we created in step 3.
9. Now we’re at line 10. We define a new variable sum in the global execution context. Temporary assignment is undefined.
10. We need to execute a function next. Which function? The function that is defined in the variable named adder. We look it up in the global execution context, and sure enough we find it. It’s a function that takes two parameters.
11. Let’s retrieve the two parameters, so we can call the function and pass the correct arguments. The first one is the variable val, which we defined in step 1, it represents the number 7, and the second one is the number 8.
12. Now we have to execute that function. The function definition is outlined lines 3–5. A new local execution context is created. Within the local context two new variables are created: a and b. They are respectively assigned the values 7 and 8, as those were the arguments we passed to the function in the previous step.
13. Line 4. A new variable is declared, named ret. It is declared in the local execution context.
14. Line 4. An addition is performed, where we add the content of variable a and the content of variable b. The result of the addition (15) is assigned to the ret variable.
15. The ret variable is returned from that function. The local execution context is destroyed, it is removed from the call stack, the variables a, b and ret no longer exist.
16. The returned value is assigned to the sum variable we defined in step 9.
17. We print out the value of sum to the console.

As expected the console will print 15. We really go through a bunch of hoops here. I am trying to illustrate a few points here. First, a function definition can be stored in a variable, the function definition is invisible to the program until it gets called. Second, every time a function gets called, a local execution context is (temporarily) created. That execution context vanishes when the function is done. A function is done when it encounters return or the closing bracket }.

Finally, a closure

Take a look a the next code and try to figure out what will happen.

 1: function createCounter() {
 2:   let counter = 0
 3:   const myFunction = function() {
 4:     counter = counter + 1
 5:     return counter
 6:   }
 7:   return myFunction
 8: }
 9: const increment = createCounter()
10: const c1 = increment()
11: const c2 = increment()
12: const c3 = increment()
13: console.log('example increment', c1, c2, c3)

Now that we got the hang of it from the previous two examples, let’s zip through the execution of this, as we expect it to run.

1. Lines 1–8. We create a new variable createCounter in the global execution context and it get’s assigned function definition.
2. Line 9. We declare a new variable named increment in the global execution context..
3. Line 9 again. We need call the createCounter function and assign its returned value to the increment variable.
4. Lines 1–8 . Calling the function. Creating new local execution context.
5. Line 2. Within the local execution context, declare a new variable named counter. Number 0 is assigned to counter.
6. Line 3–6. Declaring new variable named myFunction. The variable is declared in the local execution context. The content of the variable is yet another function definition. As defined in lines 4 and 5.
7. Line 7. Returning the content of the myFunction variable. Local execution context is deleted. myFunction and counter no longer exist. Control is returned to the calling context.
8. Line 9. In the calling context, the global execution context, the value returned by createCounter is assigned to increment. The variable increment now contains a function definition. The function definition that was returned by createCounter. It is no longer labeled myFunction, but it is the same definition. Within the global context, it is labeledincrement.
9. Line 10. Declare a new variable (c1).
10. Line 10 (continued). Look up the variable increment, it’s a function, call it. It contains the function definition returned from earlier, as defined in lines 4–5.
11. Create a new execution context. There are no parameters. Start execution the function.
12. Line 4. counter = counter + 1. Look up the value counter in the local execution context. We just created that context and never declare any local variables. Let’s look in the global execution context. No variable labeled counter here. Javascript will evaluate this as counter = undefined + 1, declare a new local variable labeled counter and assign it the number 1, as undefined is sort of 0.
13. Line 5. We return the content of counter, or the number 1. We destroy the local execution context, and the counter variable.
14. Back to line 10. The returned value (1) gets assigned to c1.
15. Line 11. We repeat steps 10–14, c2 gets assigned 1 also.
16. Line 12. We repeat steps 10–14, c3 gets assigned 1 also.
17. Line 13. We log the content of variables c1, c2 and c3.

Try this out for yourself and see what happens. You’ll notice that it is not logging 1, 1, and 1 as you may expect from my explanation above. Instead it is logging 1, 2 and 3. So what gives?

Somehow, the increment function remembers that counter value. How is that working?

Is counter part of the global execution context? Try console.log(counter) and you’ll get undefined. So that’s not it.

Maybe, when you call increment, somehow it goes back to the the function where it was created (createCounter)? How would that even work? The variable increment contains the function definition, not where it came from. So that’s not it.

So there must be another mechanism. The Closure. We finally got to it, the missing piece.

Here is how it works. Whenever you declare a new function and assign it to a variable, you store the function definition, as well as a closure. The closure contains all the variables that are in scope at the time of creation of the function. It is analogous to a backpack. A function definition comes with a little backpack. And in its pack it stores all the variables that were in scope at the time that the function definition was created.

So our explanation above was all wrong, let’s try it again, but correctly this time.

1: function createCounter() {
 2:   let counter = 0
 3:   const myFunction = function() {
 4:     counter = counter + 1
 5:     return counter
 6:   }
 7:   return myFunction
 8: }
 9: const increment = createCounter()
10: const c1 = increment()
11: const c2 = increment()
12: const c3 = increment()
13: console.log('example increment', c1, c2, c3)

1. Lines 1–8. We create a new variable createCounter in the global execution context and it get’s assigned function definition. Same as above.
2. Line 9. We declare a new variable named increment in the global execution context. Same as above.
3. Line 9 again. We need call the createCounter function and assign its returned value to the increment variable. Same as above.
4. Lines 1–8 . Calling the function. Creating new local execution context. Same as above.
5. Line 2. Within the local execution context, declare a new variable named counter. Number 0 is assigned to counter. Same as above.
6. Line 3–6. Declaring new variable named myFunction. The variable is declared in the local execution context. The content of the variable is yet another function definition. As defined in lines 4 and 5. Now we also create a closure and include it as part of the function definition. The closure contains the variables that are in scope, in this case the variable counter (with the value of 0).
7. Line 7. Returning the content of the myFunction variable. Local execution context is deleted. myFunction and counter no longer exist. Control is returned to the calling context. So we are returning the function definition and its closure, the backpack with the variables that were in scope when it was created.
8. Line 9. In the calling context, the global execution context, the value returned by createCounter is assigned to increment. The variable increment now contains a function definition (and closure). The function definition that was returned by createCounter. It is no longer labeled myFunction, but it is the same definition. Within the global context, it is called increment.
9. Line 10. Declare a new variable (c1).
10. Line 10 (continued). Look up the variable increment, it’s a function, call it. It contains the function definition returned from earlier, as defined in lines 4–5. (and it also has a backpack with variables)
11. Create a new execution context. There are no parameters. Start execution the function.
12. Line 4. counter = counter + 1. We need to look for the variable counter. Before we look in the local or global execution context, let’s look in our backpack. Let’s check the closure. Lo and behold, the closure contains a variable named counter, its value is 0. After the expression on line 4, its value is set to 1. And it is stored in the backpack again. The closure now contains the variable counter with a value of 1.
13. Line 5. We return the content of counter, or the number 1. We destroy the local execution context.
14. Back to line 10. The returned value (1) gets assigned to c1.
15. Line 11. We repeat steps 10–14. This time, when we look at our closure, we see that the counter variable has a value of 1. It was set in step 12 or line 4 of the program. Its value gets incremented and stored as 2 in the closure of the increment function. And c2 gets assigned 2.
16. Line 12. We repeat steps 10–14, c3 gets assigned 3.
17. Line 13. We log the content of variables c1, c2 and c3.

So now we understand how this works. The key to remember is that when a function gets declared, it contains a function definition and a closure. The closure is a collection of all the variables in scope at the time of creation of the function.

You may ask, does any function has a closure, even functions created in the global scope? The answer is yes. Functions created in the global scope create a closure. But since these functions were created in the global scope, they have access to all the variables in the global scope. And the closure concept is not really relevant.

When a function returns a function, that is when the concept of closures becomes more relevant. The returned function has access to variables that are not in the global scope, but they solely exist in its closure.

Not so trivial closures

Sometimes closures show up when you don’t even notice it. You may have seen an example of what we call partial application. Like in the following code.

let c = 4
const addX = x => n => n + x
const addThree = addX(3)
let d = addThree(c)
console.log('example partial application', d)

In case the arrow function throws you off, here is the equivalent.

let c = 4
function addX(x) {
  return function(n) {
     return n + x
  }
}
const addThree = addX(3)
let d = addThree(c)
console.log('example partial application', d)

We declare a generic adder function addX that takes one parameter (x) and returns another function.

The returned function also takes one parameter and adds it to the variable x.

The variable x is part of the closure. When the variable addThree gets declared in the local context, it is assigned a function definition and a closure. The closure contains the variable x.

So now when addThree is called and executed, it has access to the variable x from its closure and the variable n which was passed as an argument and is able to return the sum.

In this example the console will print the number 7.

Conclusion

The way I will always remember closures is through the backpack analogy. When a function gets created and passed around or returned from another function, it carries a backpack with it. And in the backpack are all the variables that were in scope when the function was declared.

If you enjoyed reading this, don’t forget the applause. 👏
Thank you.

I never understood JavaScript closures was originally published in DailyJS on Medium, where people are continuing the conversation by highlighting and responding to this story.

Sometimes, when you really want to understand something, it’s worth trying to build it yourself. That is what I do with promises in this article. I am trying to build a very basic ‘Promise’.

At first there were callbacks …

Let’s begin with the basics, before anybody heard of promises, we had callbacks. Take a look at the following snippet.

const testUrl = 'https://api.github.com/users/odemeulder'
const fetchData = (url, callback) => {
  let xhr = new XMLHttpRequest()
  xhr.onreadystatechange = () => {
    if (xhr.readyState == 200 ) { 
      callback(xhr.responseText)
    }
  }
  xhr.open(url)
  xhr.send()
}
fetchData(testUrl, response => console.log(response) )

What does it do? It creates a very simple function that takes a url, reads it using the XMLHttpRequest and allows a callback function to do something with the result. And in the last line, we call our new function; as a callback we provide a function that prints out the response. Very simple. If you are not familiar with XMLHttpRequest, don’t fret, it is not important to understand for the rest of the article. Just know that it is the object that the browser provides to JavaScript to perform http requests. It is an asynchronous function, it takes a callback when the onreadystatechange event fires.

What does a promise look like?

Before diving into writing our own promise, let’s see what the end result should look like. What is the call signature?

function fetchDataReturningPromise(url) {
  // what happens here?
}

function functionWithDelay(arg) {
  // what happens here?
}

fetchDataReturningPromise(url).then(response => console.log(response))
// or
functionWithDelay(arg1).then(performFollowUpOperation)
// or
functionWithDelay(arg1).then((resolvedValue) =>
  PerformFollowUpOperation(resolvedValue))

What do we have here? Let’s focus on the third example.

We have a function with a delay; something inside the function makes an asynchronous call. It could be a timeout, an http request, a database call.

That function returns ‘something’. That something has a then method, so ‘something’ must be an object. In fact ‘something’ will be a function.

The then function takes one argument, in the form of a function or a callback. And the callback passed into the then function does not get executed until functionWithDelay is done. In our third example the function we pass to then is labeled performFollowUpOperation. Note that performFollowUpOperation takes one argument (resolvedValue) and that argument is provided by functionWithDelay. Lots to unpack here. Pause, and re-read previous paragraph (if you’d like to).

In summary:

1. Define a function with a next function.

2. Define a function with a delay that returns a function with a next function (functionWithDelay)

3. The function with a next function needs to take another function as an argument. That follow-up function cannot be executed until the function with delay is done. (performFollowUpOperation)

4. The function with delay must somehow pass a value to the follow-up function.

So now we need to fill in functionWithDelay, first we need to define that thing that has a next function.

First naieve attempt

Let’s make a first attempt.

 1: function FunctionWithAThenFunction() {
 2:  this.then = function (followUpFunction) {
 3:     followUpFunction()
 4:   }
 5: }

 6: function functionWithDelay(param1) {
 7:   let xhr = new XMLHttpRequest()
 8:   xhr.onreadystatechange = () => {
 9:     if (xhr.readyState == 200) callback(xhr.responseText)
10:   }
11:   xhr.open(param1)
12:   xhr.send()
13:   return new FunctionWithAThenFunction()
14: }

All right, we have a function with a then function. Our functionWithDelay returns a function with a then function. But. What is callback on line 9? Somehow that followUpFunction should end up in the callback. That is not happening. Therefor, the followUpFunction is going to execute right away and not wait until functionWithDelay is done.

How do we get that followUpFunction to be callback?

Second attempt

Let’s take a look at functionWithDelay. What is characteristic about a function with a delay? Or what is characteristic about asynchronous functions? They will often take a callback in some way. So let’s try to re-write that function as accepting a callback, such that we can access that callback easily.

And then let’s rewrite our FunctionWithAThenFunction and let it accept an function, more specifically an asynchronous function with callback.

1: function FunctionWithAThenFunction(aFnWithDelayAndACallback){
 2:   let nextThingToDo
 3:   this.then = function ( followUpFunction) {
 4:     nextThingToDo = followUpFunction
 5:   }
 6:   aFnWithDelayAndACallback( nextThingToDo )
 7: }
 8: function functionWithDelay(param1) {
 9:   const httpRequestWithCallback = function (callback) {
10:     let xhr = XMLHttpRequest()
11:     xhr.onreadystatechange = () => {
12:       if (xhr.readyState == 200) callback(xhr.responseText)
13:     }
14:     xhr.open(param1)
15:     xhr.send()
16:   }
17:   return new FunctionWithAThenFunction(httpRequestWithCallback)
18: }
19: functionWithDelay('some argument').then(console.log)

Lines 1–7 define our FunctionWithAThenFunction. That is the function that will be returned by the function with a delay. And it expects a function as an argument. More specifically a function with a callback. The then function then sets the callback in a local variable (nextThingToDo). And that function really does one thing and that is to call the function with the callback, in our example the function labeled aFnWithDelayAndACallback.

Lines 8–18 define our functionWithDelay. In our example the function that does an http request and returns something with a then function. And that something with a then function is a new instance of FunctionWithAThenFunction. It looks a little different than what we did above. We create a new local temporary function named httpRequestWithCallback, which accepts a callback argument and wraps the entire XMLHttpRequest code from the prior examples. That new local function is then passed as an argument to FunctionWithAThenFunction.

Now, if you try to run this (line 19), it will still not accomplish what we are trying to do. If you follow along, you will see that nextThingToDo is undefined as we are calling it before we get a chance to assign it a value. We are missing one thing and that is some kind of status.

Third Attempt

Let’s add a variable that keeps track of the status of execution.

1: function FunctionWithAThenFunction(aFnWithDelayAndACallback){
 2:   let nextThingToDo
 3:   let status = 'pending'
 4:   this.then = function ( followUpFunction) {
 5:     if (status === 'pending') {
 6:       nextThingToDo = followUpFunction
 7:     }
 8:     else {
 9:       followUpFunction()
10:     }
11:   }
12:   aFnWithDelayAndACallback( (newValue) => { 
13:      status = 'resolved'
14:      nextThingToDo(newValue)
15:   })
16: }
17: function functionWithDelay(param1) {
18:   const httpRequestWithCallback = function (callback) {
19:     let xhr = XMLHttpRequest()
20:     xhr.onreadystatechange = () => {
21:       if (xhr.readyState == 200) callback(xhr.responseText)
22:     }
23:     xhr.open(param1)
24:     xhr.send()
25:   }
26:   return new FunctionWithAThenFunction(httpRequestWithCallback)
27: }
28: functionWithDelay('some argument').then(console.log)

You see that we declare a status variable (line 3). This way we cover all of our basis. If then is called before the main function is done, we store the followUpFunction in a variable, if not we can safely execute it. But it does not have the value from the main function. Let’s fix that.

1: function FunctionWithAThenFunction(aFnWithDelayAndACallback){
 2:   let nextThingToDo, value
 3:   let status = 'pending'
 4:   this.then = function ( followUpFunction) {
 5:     if (status === 'pending') {
 6:       nextThingToDo = followUpFunction
 7:     }
 8:     else {
 9:       followUpFunction(value)
10:     }
11:   }
12:   aFnWithDelayAndACallback( (newValue) => { 
13:      status = 'resolved'
14:      value = newValue
15:      nextThingToDo(newValue)
16:   })
17: }
18: function functionWithDelay(param1) {
19:   const httpRequestWithCallback = function (callback) {
20:     let xhr = XMLHttpRequest()
21:     xhr.onreadystatechange = () => {
22:       if (xhr.readyState == 200) callback(xhr.responseText)
23:     }
24:     xhr.open(param1)
25:     xhr.send()
26:   }
27:   return new FunctionWithAThenFunction(httpRequestWithCallback)
28: }
29: functionWithDelay('some argument').then(console.log)

Here we declare a new local variable named value. (line 2) When we resolve the main function we set that value. So it can be passed as a parameter to followUpFunction on line 9.

Speaking of resolving, let’s add re-write this for clarity. No change in functionality with the previous sample.

1: function FunctionWithAThenFunction(aFnWithDelayAndACallback){
 2:   let nextThingToDo, value
 3:   let status = 'pending'
 4:   this.then = function ( followUpFunction) {
 5:     if (status === 'pending') {
 6:       nextThingToDo = followUpFunction
 7:     }
 8:     else {
 9:       followUpFunction(value)
10:     }
11:   }
12:   const resolve = newValue => {
13:     value = newValue
14:     status = 'resolved'
15:     if (nextThingToDo) nextThingToDo(newValue)
16:   }
17:   aFnWithDelayAndACallback(resolve)
17: }
18: function functionWithDelay(param1) {
19:   const httpRequestWithCallback = function (callback) {
20:     let xhr = XMLHttpRequest()
21:     xhr.onreadystatechange = () => {
22:       if (xhr.readyState == 200) callback(xhr.responseText)
23:     }
24:     xhr.open(param1)
25:     xhr.send()
26:   }
27:   return new FunctionWithAThenFunction(httpRequestWithCallback)
28: }
29: functionWithDelay('some argument').then(console.log)

Here we introduce the local resolve function for clarity on lines 12–16. It is subsequently called on line 17.

Putting it all together

We’ve come a long way. Let’s re-write the last example one more time and change the function names a little bit.

• FunctionWithAThenFunction becomes Promise
• functionWithDelay becomes fetchData
• httpRequestWithCallback becomes httpRequest
• aFnWithDelayAndACallback becoms fn

function Promise(fn){
   let nextThingToDo, value
   let status = 'pending'
   this.then = function ( callback ) {
     if (status === 'pending') {
       nextThingToDo = callback
     }
     else {
       callback(value)
     }
   }
   const resolve = newValue => {
     value = newValue
     status = 'resolved'
     if (nextThingToDo) nextThingToDo(newValue)
   }
   fn(resolve)
 }
function fetchData(url) {
  const httpRequest = function (callback) {
    let xhr = XMLHttpRequest()
    xhr.onreadystatechange = () => {
      if (xhr.readyState == 200) callback(xhr.responseText)
    }
    xhr.open(url)
    xhr.send()
  }
  return new Promise(httpRequest)
}

const testUrl = 'https://api.github.com/users/odemeulder'
fetchData(testUrl).then(console.log)

The Promise code can now be used in other functions. Let’s say you wanted to have a function similar to setTimeout that returned a promise. Let’s call the function Wait and it would accept a delay time in milliseconds.

function Wait(milliSeconds) {
  const fn = (callback) => setTimeout(callback, milliSeconds)
  return new Promise(fn)
}
Wait(5000).then(() => console.log('we waited 5 seconds'))

Or let’s say you want to use the httpRequest example in Node. In Node XmlHttpRequest won’t work because that is an object provided by the browser, you’d have to use Node’s built in https module.

const https = require('https')
const testUrl = 'https://api.github.com/users/odemeulder'
function fetchDataForNode(url) {
  const fn = callback => {
    const options = {
      url: url,
      headers: { 'User-Agent': 'request' }
    }
    https.get(options, response => {
      let ret
      response.on('data', chunk => ret += chunk)
      response.on('end', () => callback(ret))
    })
  return new Promise(fn)
}
fetchDataForNode(testUrl).then(console.log)

Again, don’t fret over the technicalities of how https.get works in node. The important part is: create a function that takes a callback and wraps an asynchronous call, and pass that function to your Promise constructor.

Next Steps

There is obviously much more to a promise than what we just outlined. This article is long enough already. But here are some improvements / additions.

One thing is that the then function should return a new promise. That allows you to chain promises. doSomething().then(somethingElse).then(aThirdThing)

And the other big thing that we did not go into is error handling. Our Promise function should also implement a catch function. That function would take a function as a parameter and that function would be called in case of any errors.

Conclusion

We created a very basic Promise object, and I showed you three examples of how to create a function that returns a promise (fetchData for browser http requests, fetchDataForNode for http requests in Node and wait). In each case, you create a function that accepts a callback, create a new Promise function and pass the function as callback.

I welcome any feedback as well as applause.

I Promise you a Miracle was originally published in HackerNoon.com on Medium, where people are continuing the conversation by highlighting and responding to this story.

This post is NOT a post to show you how to use forEach, map, filter and reduce, NOR is it a post showing the benefits of higher order functions. There are plenty of other articles that do that.

When I want to understand a basic concept, I like to try to implement it myself, even if it is a very basic version. People learn in different ways, this works for me. So let me take you on this journey as I try to create my own forEach, map, reduce and filter functions in JavaScript.

myForEach

In order to disambiguate from the built-in forEach, let’s call my new function myForEach.

First thing to do is to look at the call signature. So, the forEach function you know does something like this:

let arr = [ 1, 2, 3, 4 ]

// log each item of the array
arr.forEach(console.log)

// double each item of the array and log it to the console
arr.forEach((val) => console.log(val * 2))

We’re sort of looking for a function that takes two parameters, an array and callback function. The callback function could be a function definition or an anonymous function. And what we want this to do is to iterate over each item in the array and call the callback on that item. So let’s give it a shot.

function myForEach(arr, callback) {
  for (let i = 0; i < arr.length; i++) {
    callback(arr[i])
  }
}

let arr = [ 1, 2, 3, 4 ]

// log each item of the array
myForEach(arr, console.log)

// double each item of the array and log it to the console
myForEach(arr, val => console.log(val * 2))

Seems simple enough and it works. But … but … it’s not exactly the same. If you try arr.myForEach(console.log), you get TypeError: arr.myForEach is not a function. That is because we declared myForEach as a global function. And really, it should be part of the prototype of the Array object. Note the purpose of this article is not to explain how prototypical inheritance works. In order to achieve what we want, we need to do this as follows.

Array.prototype.myForEach = (callback) => {
  for (let i = 0; i < this.length; i++) {
    callback(arr[i])
  }
}

// log each item of the array
arr.myForEach(console.log)

When I tried running the code above, nothing happened. What is wrong?

Trying to get to fancy with arrow functions. Arrow functions don’t have their own this or arguments binding. So when I call this inside the function definition, it does not refer to the Array object itself, as I naively expected. Here is the correct way to do it. This StackOverflow post has the explanation.

Array.prototype.myForEach = function (callback) {
  for (let i = 0; i < arr.length; i++) {
    callback(arr[i])
  }
}

// log each item of the array
arr.myForEach(console.log)

Beautiful. Now that we have the mechanics down, we can move on to the other higher order functions.

myMap

The signature is as follows. Imagine we have an array of numbers, and want to get an array of the squares of these numbers.

let arr = [ 1, 2, 3, 4 ]

let arrNew = arr.map(val => val * val)

// or 
const squareNumber = n => n * n
let arrNew2 = arr.map(squareNumber)

console.log(arrNew)
// outputs [ 1, 4, 9, 16 ]

So we we apply to an array: a function that takes a callback, and returns a new array. The callback is a function that takes an input value and usually transforms that input value into something else.

Array.prototype.myMap = function(callback) {  
  let ret = []
  for (var i = 0; i < this.length; i++) {
    let newVal = callback(this[i])
    ret.push(newVal)
  }
  return ret
}

We iterate through the array (represented by this), apply the callback function, and push the resulting value onto a new array. And subsequently return that new array.

We could write this more succinctly using our new myForEach function.

Array.prototype.myMap = function(callback) {  
  let ret = []
  this.myForEach(val => ret.push(callback(val)))
  return ret
}

Moving on.

myFilter

By now we’re starting to get the hang of it. Here is the filter call signature.

let people = [ 
  { 'name': 'Bob', 'age': 70 },
  { 'name': 'Sue', 'age': 30 },
  { 'name': 'Joe', 'age': 18 } ]

let youngPeople = people.filter( person => person.age < 69 )
// returns array with two objects.
//  [ { 'name': 'Sue', 'age': 30 },
//  { 'name': 'Joe', 'age': 18 } ]

// or
const isYoung = person => person.age < 59
let youngPeople = people.filter(isYoung)

So the filter expects a callback which returns a true or false, also referred to as a predicate. The function returns a new array. Based on our experience we can quickly whip this up.

Array.prototype.myFilter = function(callback) {
  let ret = []
  this.myForEach( (val) => {
    if (callback(val) === true) ret.push(val)
  }
  return ret
}

Pretty straight forward. Note I could have written if (callback(val)) instead of if (callback(val) === true), but I did this to emphasize that we expect the callback to return true or false.

myReduce

The reduce function is a little more complex to grok. The textbook example of reduce is to sum elements of an array. This is what it would look like.

let arr = [ 1, 2, 3, 4 ]

let sum = arr.reduce( (acc, val) => acc + val )

console.log(sum)
// prints out 10

The callback takes two arguments, the first one is often referred to as the accumulator, and the second one is the value of a particular item of the array. In this example, the accumulator keeps a running total, and with each iteration over the array, its value is added to the accumulator. And at the end the accumulator contains the value we are trying to get.

This is what the function looks like in its most basic form.

Array.prototype.myReduce = function (callback) {
  let ret
  this.myForEach( val => {
    if (ret === undefined) ret = val
    else ret = callback(ret, val)
  }
  return ret
}

It’s a little trickier than the other higher order functions we defined above. Note how we need to check of ret is undefined or not. This brings up another point. The JavaScript reduce function lets you specify an initial value. That’s not hard to add to our function. We just add an init parameter to the function definition and set the ret variable to init initially.

Array.prototype.myReduce = function (callback, init) {
  let ret = init
  this.myForEach( val => {
    if (ret === undefined) ret = val
    else ret = callback(ret, val)
  }
  return ret
}

const sumTwoValues = (a, b) => a + b

let sum = arr.myReduce( sumTwoValues , 0 ) // added 0 as an argument for initial value.

console.log(sum)
// prints 10

Improvements

For each of the functions above, we could make it such that we include the index of the array element in the callback signature and make it available in the callback. This should allow us to do the following for example.

var arr = [ 1, 2, 3, 4, 5, 6 ]

arr.forEach( (val, idx) => { if (idx % 2 === 0) console.log(val) }
// prints the value of every other item in the array.

arr.map( (val, idx) => idx % 2 === 0 ? val * 2 : val * 3 )
// create a new array where every even item has been doubled and every odd item has been trippled

arr.map( (val, idx) => idx == 0) 
// return an array with only the first item 

The implementation is quite easy. In our myForEach function, we pass the value and the index to the callback. And since we use myForEach in all the other functions, we can easily include the index in the callback call.

Array.prototype.myForEach = function(callback) {
  for (var i = 0; i < this.length; i++) {
    callback(this[i], i)
  }
}

Array.prototype.myMap = function(callback) {
  let ret = []
  this.myForEach((val, idx) => ret.push(callback(val, idx)))
  return ret
}

Array.prototype.myFilter = function(callback) {
  let ret = []
  this.myForEach((val, idx) => { if (callback(val, idx)) ret.push(val)} )
  return ret
}

Array.prototype.myReduce = function(callback, init) {
  let ret = init
  this.myForEach( ( val, idx ) => {
    if (ret === undefined) ret = val
    else ret = callback(ret, val, idx)
  })
  return ret
}

One more thing we should do is some error checking. For instance we should make sure that the callback is a function. We should check that the array is not null.

Array.prototype.myForEach = function(callback) {
  if (this == null) {
    throw new TypeError('array is null')
  }
  if (typeof callback !== 'function') {
      throw new TypeError(callback + ' is not a function'
  }
  for (var i = 0; i < this.length; i++) {
    callback(this[i], i)
  }
}

Array.prototype.myMap = function(callback) {
  if (this == null) {
    throw new TypeError('array is null')
  }
  if (typeof callback !== 'function') {
      throw new TypeError(callback + ' is not a function'
  }
  let ret = []
  this.myForEach((val, idx) => ret.push(callback(val, idx)))
  return ret
}

Array.prototype.myFilter = function(callback) {
    if (this == null) {
    throw new TypeError('array is null')
  }
  if (typeof callback !== 'function') {
      throw new TypeError(callback + ' is not a function'
  }
  let ret = []
  this.myForEach((val, idx) => { if (callback(val, idx)) ret.push(val)} )
  return ret
}

Array.prototype.myReduce = function(callback, init) {
  if (this == null) {
    throw new TypeError('array is null')
  }
  if (typeof callback !== 'function') {
      throw new TypeError(callback + ' is not a function'
  }
  let ret = init
  this.myForEach( ( val, idx ) => {
    if (ret === undefined) ret = val
    else ret = callback(ret, val, idx)
  })
  return ret
}

That’s the end. The actual Javascript forEach, map, filter and reduce functions are a more complex. But by coding this base implementations, you may better understand how they work under the hood.

If you’ve enjoyed it, send me some love and click the heart below.

Higher Order Functions: Behind the Scenes was originally published in HackerNoon.com on Medium, where people are continuing the conversation by highlighting and responding to this story.

A corporation is run by a Chief Executive Officers and other C-level officers that report to him: Chief Financial Officer, Chief Operations Officer, Chief Marketing Officer, Chief Technical Officer. Sometimes, a company is so big, that the corporation is organized around different business units. In that case, each business unit has it’s own CEO, COO, CFO, … It may have different names, but let’s refer to the CEO and all the people that report to him as the Executive Team.

Now imagine that a catastrophe hits.

The CEO and it’s entire Executive Team are all involved in a major scandal. And not only them, but also all the people that report to directly to them. So let’s say we have a CEO and 10 members of the Executive Team. And each of these people have 10 people that report directly to them. So we’re talking a hypothetical 111 people. All of them, every single one is involved in the scandal. And. They. All. Get. Fired.

No one left at the top.

The board jumps in and they quickly, within months, replace (almost) every vacant position. Since they don’t want anybody that was involved with the previous leadership team, every position is filled with someone coming from the outside.

In the beginning it’s quite chaotic. After a year, every position is filled. People get used to the new leadership. The new leadership makes mistakes. Because 1. Everybody makes mistakes and 2. They’re new at this. But they learn from their mistakes. They get better at what they do. And finally after 3 or 4 years, some sense of normalcy returns.

And now imagine, that this same pattern would repeat itself like clockwork every 4 years. Every four years, the entire leadership of the corporation is fired and replaced with new blood.

How could a corporation function like that? Could it even?

And now you probably get what I am trying to get at. This is what we do with our government. It’s especially interesting to see at the federal government.

Every four years (or 8 years) we have a new president, new cabinet secretaries and new deputy cabinet secretaries. They all get appointed from outside the agency that they’re appointed to. Sometimes the vision of the government or agency totally changes.

And note that the federal government is way bigger than a Fortune 500 corporation.

So how can it function? The fact that he government functions at all is probably a small miracle, considering what I outlined above.

I thought about this and came to this conclusion. In the beginning I wrote “Such a company must have a bureaucracy, a set of systems and procedures, a reporting structure to make all their employees work in tandem towards a common goal.” I the case of government the reporting structure changes fundamentally every 4 years and the common goal may change as well. So all that is left are the bureaucracy and the systems and procedures. The proverbial red tape is all that keeps the government running.

Not sure where I am trying to go with this story. I just thought it was an interesting thought experiment. Imagine a major corporation or organization, whose leadership team gets eviscerated every four years. That organization is running the country.

Fire the entire leadership team was originally published in Extra Newsfeed on Medium, where people are continuing the conversation by highlighting and responding to this story.