I recently had someone respond to an article I wrote and say, among other things, that if your team is in a situation where you realize that the cloud provider and service you choose isn’t the best fit and you need to switch… it’s your own fault.

…the “oh I’m not getting the performance I need from this provider” is a failure of your engineers to vet the services they are using for your use cases.

To be fair, I like to hold engineers to a high bar, so I don’t completely disagree. However, it’s not reasonable, especially in the context of an early-stage startup, to expect engineers to look into the future and predict through the next two years what will stay the same, what will change, and what those changes will be.

I wrote an article about senior engineers and what it means to be one. The point I make is that the most senior engineers understand one thing: it’s not possible to be absolutely right. Even if you look at all the factors, collect all the data, and understand every facet of the problem, things change. You can’t predict change. It’s all the more true with infrastructure.

Let’s assume for a moment that I’m wrong. Here are the factors, based on my personal experience of what has led to infrastructure changes in my career, that you would have to predict to be right about your infrastructure choices for more than one to two years in an early-stage startup environment:

1. The Market
2. The Product
3. The Team
4. The Users
5. The Budget

The Market

Sometimes, as a startup, you have to react to the market. It’s possible that interest rates go up, conflict breaks out in the Middle East, or a regime collapses in South America and — somehow — due to that, you have to modify your product to support a new feature. It sounds crazy, but startups are crazy.

Something as simple as an industry leader acquiring an up-and-comer and then immediately sunsetting their service can drive your users to start screaming, “take my money” if you’ll only support the use case the sunsetting product solved for.

More likely is, a competitor modifies their product to do what you’re doing with twice the performance and half the wait time for end-users. Your customers begin leaving your product for the competitors because the value of that time saved is too high to ignore.

All of this can lead to infrastructure changes. You might need to use a different cloud service, possibly on another cloud provider, that has lower concurrency restrictions, higher memory limits, lower cost, or better performance for your specific new use case.

The Product

In the search for that holy grail that we call product-market-fit, it’s not unusual for your product to turn out completely different from what it started as initially. If it did change, that’s a good thing. If you have the humility to listen to the market and change your product to fit the needs that exist, you’re ahead of the game.

You might start a dating site and end up with a video blogging platform, start an MMORPG game and end up with an online instant messaging platform, or start a social-good network and end up with a daily deals app. All this can happen in a year or two.

In any one of those changes, your infrastructure will have to change as well. As a founder, if you believe your initial problem/solution idea will last forever and commit fully — from a technical perspective — to a specific infrastructure provider and service, then you might be just a little bit screwed when you need to make large changes. In this context, screwed means you have to spend exceeding amounts of time and money. Some companies, like YouTube, Slack, and Groupon made it work (the examples above). Do you have the time, budget, and energy to work hard enough to push their luck?

The Team

Sometimes, at an early-stage startup, you have one person, out of the massive three-person company, who is the sole infrastructure expert. Many teams refer to this as the bus factor, and good teams will seek to spread that expertise around when the bus factor is less than three. But at a startup, there’s little you can do. Now, what happens when the bus comes and takes your one infrastructure expert off to a seven-figure job at Google?

Hiring isn’t easy or cheap. You might find someone with the same infrastructure expertise as the member who was ‘hit’ by the bus. If you can’t, you might need to make changes when your new expert, who has different values and tastes than the previous one, insists things will have to change to continue.

On the other hand, what if a bus pulls up and drops off a new team member? The new team member has ten years of experience more than the team member who initially designed your infrastructure. The latest expert says you can get 2x the performance at half the cost if you make significant changes.

Would you spend a week of dev time to cut your monthly cloud cost in half? Would you spend a day?

The Users

Users: those beautiful, sweet, wonderful, annoying, painful, awful little things that keep us going; they can surprise you, and their behaviors can change. At an early-stage startup where every $1 of revenue is worth a gallon of blood, sweat, and tears when customers begin complaining about platform stability, you will do whatever you need to, no matter how drastic, to satisfy them.

I’ve done a lot of platform performance and site reliability work, and when it comes to users, what we’re concerned about is their behavior. As your product grows, their behavior can change. Many times, as you get more users, you start to see distinct behaviors that weren’t clear when you only had one — or none.

Teamflow is an example. It’s a Slack meets Zoom company that provides a virtual office product. Users can design a virtual office, sit in it, move through virtual space to their coworker’s office, knock on the door, and get an all-around real-world experience while working remotely.

When the product was just getting started, we chose to use GCP App Engine. At the time, it was the best choice, the most cost-effective and performant service available to us, given our specific expertise. It also allowed us to build rapidly because of the simplicity. As we grew and got more users, we noticed some incredibly spikey user behavior. Every morning, at 7:59 AM, there would be zero users on the platform. Two minutes later, at 8:01 AM, nearly 80–90% of our daily active users were online. App Engine couldn’t keep up with that kind of spikey scaling. For the first hour of every weekday, our platform was nearly unusable. We moved to Cloud Run. It wasn’t easy, but we didn’t have a choice.

The budget

When the runway starts to run out, people start to ask very deep questions about the infrastructure. It’s typically the largest cost after salaries, so it gets the side eye first.

Even if the runway isn’t running out, nobody likes to leave money on the table. I talked to a founder yesterday who has $1M in free credits available across the three major cloud providers (AWS, GCP, and Azure), but his infrastructure is stuck on AWS where he can’t make use of the other credits. As a seed-stage startup, he can’t afford to take a week off to move his infrastructure.

Sometimes, in the beginning, we make sub-par choices because of budget constraints. When the budget increases because your startup is succeeding and growing, you’ll want to take that corner-cutting infrastructure choice you made and sharpen it up. Depending on how it was built, you might be screwed.

Again, screwed means it will cost you an exorbanate amount of time and money to make changes.

Conclusion

If you still think you can make infrastructure choices that are right in the present and will be right in the future… I can’t help you, good luck. If you see what I see, you’re realizing there’s no hope; you can’t predict varying factors moving unpredictably through time. It’s time to consider your ability and capacity to make infrastructure changes.

I write my code in an infrastructure-agnostic way. I call it cloud-nimble. I don’t depend directly on cloud APIs (like AWS S3), and within my build pipeline, I reduce my services to their smallest individual unit: functions. When I need to make changes, I can move a function, or a collection of functions, to a serverful service, a serverless service, or do a lot of other dynamic things with them.

I’m packaging all this up, building out a suite of developer tools to make it easy for others to use, and offering it as a platform: Exobase.

Share Your Thoughts: Agree or Disagree?

If you’ve had to move infrastructure in the past, I’d love to hear about your experience. What factors made you move?

If you’ve never had to move infrastructure in the past, how did you beat the odds?

If you’re mad this article ended in marketing, then I’ll assume you disagree with the basic premise that there is a problem with infrastructure being hard to move… is your infrastructure easy to move?

if (reader.read(this)) {
  request(reader, 'comment')
}

Startup Infrastructure: No Matter the Choice, You’re Still Wrong. was originally published in Exobase on Medium, where people are continuing the conversation by highlighting and responding to this story.

Assuming your infrastructure is on AWS, what would happen if…

1. AWS claims your platform breaks its policies and shuts down your account.
2. AWS’s internal security system identifies your customers’ requests to your API/platform as a D/DOS attack and begins rate-limiting those requests.
3. You run out of free credits, you have limited runway, and you haven’t yet generated enough revenue to cover your cloud costs.
4. As your customer base grows, you discover your chosen cloud service can’t handle your uniquely spikey workload at scale.
5. One of the AWS services your platform depends on has an outage. Your platform is completely unavailable to your customers.
6. A company approaches you, offering an acquisition, but only if you can move your infrastructure to GCP in the next two weeks.

For these examples, the answer is: You need to move your platform to a different cloud or cloud service. If you don’t, you’re in real trouble.

If this sounds like a list of complete impossibilities, read AWS and GCP Horror Stories. Bad Reasons Good Companies Died. These things happen.

The difference between scalable and nimble

Cloud-nimble means having the ability to quickly and easily move your platform between clouds or cloud services. If moving your platform from AWS to GCP requires code changes, you are not cloud-nimble.

How to be cloud-nimble

Cloud-nimble is being achieved in today’s market by using open-source constructs like Kubernetes. It’s a good start, but it leaves us with a lot of limitations.

Plural, Architect, and Porter allow you to be cloud-nimble, but only if you’re willing to give up using native cloud services that don’t run Kubernetes. This means that if your API runs best on AWS Lambda — in terms of cost and performance — you can’t use these platforms and you can’t be cloud-nimble.

How to be cloud-native-nimble

Today, it’s not possible to be cloud-nimble and cloud-native. Nobody has built the tools you need, but I’m working on it at Exobase.

If you'd like to get notified when Exobase is publically available or are interested in being a beta user, email me at ray@exobase.dev

Your Cloud Infrastructure Scales, but is it Nimble? was originally published in Exobase on Medium, where people are continuing the conversation by highlighting and responding to this story.

Nobody plans to leave AWS, but as they say, “sh*** happens.”

As engineers, when we write software, we’re taught to keep it nimble by never depending directly on external systems. When it comes to the cloud… *crickets*. Companies have died because they needed to move off AWS but couldn’t do it in a reasonable and cost-effective timeline.

It’s not rocket science why we do this. It’s simple: there are no alternatives. But maybe there is. I’ll explain why you should take cloud-nimble architecture seriously and then show you what I do to keep my projects cloud-nimble.

Cloud Service Optimization

The first reason you should value the ability to switch clouds and cloud services is the ability to select and use the cloud service that’s price and performance-optimized for your use case.

When I first got into serverless, I thought GCP’s Cloud Function service was better than AWS Lambda. I don’t recall why I thought that, but I built a consumer-facing API on GCP Cloud Run.

Can you guess what happened?

It was a horrible mistake. My API had an insane latency problem. Cold start requests added — at a minimum — 2 seconds per request. The AWS team has worked hard to build a service that can do things that GCP’s Cloud Functions simply can’t, specifically around cold starts and latency.

I had to move my infrastructure to a different service.

This happened again at Teamflow, a virtual office startup (think Slack meets Zoom). We built our API on GCP’s App Engine. The use case of a virtual office is naturally hyper-spikey workloads. At 7:59 AM, zero people are using your product. Three minutes later, at 8:02 AM PT, over 90% of your customers are online. Because App Engine couldn’t scale fast enough, those customers were banging the refresh button in frustration as the app failed to load.

I spent three weeks, with two other engineers, modifying our code and infrastructure to move it to GCP’s Cloud Run service — which was much easier to configure for our hyper-spikey scaling requirements.

Avoiding The Killswitch

AWS, GCP, Azure, and every other cloud provider reserves the right to ‘unalive’ your account and destroy your infrastructure anytime they want. If you’re thinking, “I’m not pot, porn, or gambling, so that would never happen to me,” you are wrong.

I recently spoke with a founder who told me a nightmare story. His team was using GCP’s Cloud Run, a container service, to host their API. They had a unique use case that required them to call back to their own API to kick off more work. It turns GCP monitors for this type of behavior and flags it as crypto mining. One sunny day, their infrastructure was gone, and their account was locked. They spent the next week working ~18 hours a day to move to AWS.

Maybe you are building a pot, porn, gambling, crypto mining, political, or firearm-related application. I don’t judge. I’m just telling you to think ahead and build like your cloud provider might kick you off their platform tomorrow.

Unlocking Free Credits

If you’re a pre-seed or seed-stage startup, free cloud credits can be the hand of mercy holding fire back from consuming your runway. Cloud credits are easy enough to get, but the way most people build their infrastructure — as though they’ve made a lifelong blood pact with their cloud provider — they’re limited to only the credits that cloud provided them.

Could you move from AWS to GCP when your credits are up?

If you’re a YCombinator startup, you can get $150K in free credits on AWS and about $200K on GCP. If you’re building AI, Azure will give you $300K in free credits to run your models on their cloud.

The ability to move your infrastructure from one cloud to another could save you an immediate $200K. Why would you ever build in a way that locked you into a single cloud?

Maximizing Redundancy

If Silicon Valley Bank can fail, so can Amazon. It may just be an outage, but the only thing certain about life and companies is that everything dies. Your job is to keep yours alive as long as possible.

How much money would your company lose during a 12-hour outage on AWS?

True disaster recovery is the ability to move from AWS to GCP. True redundancy is to run segments of your platform on AWS and GCP in parallel at all times.

Cloud Cost Negotiation

The ultimate move in any negotiation is to walk away. If the other side knows you can’t, it means their hands are wrapped firmly around a very specific organ of yours — whatever that is for you. Cost negotiations with cloud providers are no different.

Suppose you can’t leave AWS because you’ve built $100M worth of infrastructure on their platform. Suppose you’ve tightly coupled that infrastructure to their APIs (like S3, Cognito, and SQS). You can’t walk away. You’ll have to eat whatever number they tell you to regarding cost.

The ability to negotiate costs might not seem like a very big deal. If you’re a small company, it’s not. If you really did have $100M of infrastructure, then getting AWS to give you a 3% discount is substantial.

A Very Bad No Good Day

We don’t know what we don’t know. There are millions of permutations left unseen by the human eye. You never know what will happen or why. Here’s a crazy, impossible scenario: Google wants to acquire your company, but your infrastructure runs on AWS.

Meet Matt, this is his story. He did the founder’s hustle and the grind to bring a company to life from nothing. GCP approached and started talks to acquire. It was a dream come true until the conversation turned to the topic of their AWS infrastructure. Google, as you might expect, can’t acquire companies that run on AWS, and Matt’s team couldn’t move their platform to GCP in a time and cost-effective way. In the end, Google walked away, and Matt lost the exit.

You never know what can happen. Having the ability to quickly and easily switch between cloud providers might save your company or just save your company a few million dollars.

My Solution

Here’s what I do about it. I use Exobase. It started as a library that abstracted the cloud-specific APIs into generic APIs. Today, it’s a full-service build and deployment platform that allows you to deploy to AWS and GCP with just a few clicks.

Exobase isn’t public yet, but if you’re interested in being an early user, send me a message: ray@exobase.dev.

AWS and GCP Horror Stories. Bad Reasons Good Companies Died. was originally published in Exobase on Medium, where people are continuing the conversation by highlighting and responding to this story.

If you have integration tests your software is probably lacking the fundamentals of good design.

TLDR;

Integration tests are only necessary if you’ve allowed two parts of your code that are unrelated to become tightly coupled. Quality software does not tightly couple unrelated things. Therefore, integration tests are a strong indicator of poor design.

Types of Testing

There are many different types or families of tests. I typically break the modern web application testing into three common groups: unit, integration, and end-to-end.

Unit tests should test deterministic functions. Given some input to a function, you want to assert the correct output. Unit tests are the simplest to write and fastest to run. They are ideal.

As an aside, if you find that writing unit tests is hard (not simple to write), if you find yourself in a hellscape of mocking and injecting and hacking at the runtime to write unit tests you are also experiencing the consequences of poor software design.

End-to-end testing is where we test the thing in real life, running it live — whatever it is. Sometimes your unit tests pass and you expect everything to work but when you deploy your API, site, service, or app it fails completely. The real world of bootstrapping, connecting to external services, and networking can cause errors you won’t catch in unit tests (or even local manual tests). End-to-end tests ensure the thing works in the wild.

Integration tests are a way to test your so-called application logic within the structure of a framework.

If you’ve read any of my previous articles and began to drink my software purist Koolaid then the previous sentence in bold should sound off in your head like the alarms of a nuclear plant experiencing a radio-active reactor meltdown.

When you write your code inside a framework the only way to test many of your code paths is to execute the entire framework. This is why integration tests exist. They test the integration of your code with the framework. You’ll find them in any language, like a crutch, allowing engineers to lazily write code quickly in exchange for total surrender to the framework and the quality of their application.

What’s the Problem?

Quality software does not tightly couple unrelated things. Integration tests only exist to test code that was written so deeply within a framework that you can’t test it without executing the framework. That is, integration tests exist to test code you tightly coupled to an unrelated framework.

Integration tests are designed to test what some call application logic. Application logic is not a real type of logic — it is a lie. There are only three types of logic: business, utility, and access. Application logic is a dangerous term because it acts as a catch-all for all the logic in your entire application.

When you write your code with a mental model of application logic in mind you will write code where everything is attached to everything. Quality software is thoughtfully organized and uses enforceable boundaries to implement the organization.

Integration tests are a hack, a lazily wielded utility tool that allows engineers to test code they have written poorly, code that has no organization or thoughtful design except those few lines of advice they took from their chosen framework’s getting started page.

I’ve Been There

I’m not an all-knowing being of higher celestial order, I’m just an engineer who suffered through bad code long enough to open my mind and ask why. I’ve written countless integration tests. I am guilty of suggesting them, designing them, and wiring them up with great pride.

My story, going from integration test koolaid drinker to integration test hater, isn’t about the tests — it’s about the code. About three years ago I developed the Abstract & Compose design pattern. Since then, I haven’t needed to write an integration test.

You’re not a bad engineer if you are writing integration tests — you’re probably a good engineer — but your code could be better. Objectively, it is better to have business logic loosely coupled to the framework. No?

Concrete Examples

Javascript is my personal goto language so I’ll use it here but note that this applies to any language or framework.

This does not apply to Java. If you’re using Java, then the problem actually is Java and the first step is to seek help. Call 1–800-nomo-java to talk with a counselor today. Don’t throw your life away for a language that can’t love you back.

Let’s assume we’re building an API that publishes alerts. Clients send us requests, we then record and broadcast them. Whatever, we’re making it up.

This is the Express.js getting started example with some logic added to the POST /alert route.

The first thing we need to do is identify our business logic, that's what we need to test. A short list of cases to test:

• when no message is passed in the body an error should be thrown
• when no app_id is passed in the body an error should be thrown
• when a previous alert was posted within 5 seconds the alert should be skipped
• a successful alert should be added to the database
• a successful alert should be broadcast using the pubsub

Notice something, these test cases have zero relation to the framework. We’re currently using express but we could use the raw Node http module, Koa, or even AWS Lambda and these test cases would not change. However, if we did switch to AWS Lambda we would have to change our code. This is the definition of tightly coupling unrelated things.

With the current setup, we would need to use integration tests to run the entire express js application to test our business logic. Our code is literally written into the framework callback: app.post(path, our_business_logic)

With a little creativity, we can resolve it this. The naive approach would be to simply remove the business logic function and pass any dependencies to it so we can easily write tests without executing the framework.

Our new postAlert function expects a body and returns an object or throws an error. This interface is completely agnostic of the framework. You can import postAlert in a test, call it with some input, and assert some response.

This isn’t elegant, but it's a good first step to removing our code from the framework. The line where we call postAlert and pass it req.body is still problematic. We’ll want to make sure it’s tested but in order to do so, we will still have to write an integration test to execute the entire framework.

Let’s make a higher-order function that can be used for any endpoint to abstract away the framework and call it withExpress

Now, our endpoints can be wired up with complete abstraction from the framework.

All these functions ( postAlert , listAlerts , and silenceAlert ) will take one AgnosticRequest argument. The withExpress function can be tested in isolation with unit tests and each function can be tested with unit tests.

Enjoy a new world of simple unit tests!

Short Comings

We’ve accomplished the basic goal, to decouple the business logic from the framework so we can remove the need for integration tests, but it’s a naive solution at best. Our withExpress and AgnosticRequest are a good start but could be much more robust. They should handle errors, path parameters, and status codes — just to name a few.

If you like this pattern, and you want something more robust, and you don’t want to write it, check out the Abstract & Compose design pattern and the Exobase library.

I’m Ray, I created Radash, the ACP design pattern, and the Exobase library. I don’t want your money, I just want to work on codebases that don’t suck so please follow, comment, interact, and let’s make better practices together.

TLDR;

Javascript is an incredibly robust programming language with a diverse set of language features. Most of them are complete garbage. In the last 10 years, they have only served to inhibit engineers' ability to write quality code. I only use objects and functions and it’s so much better.

The Problem with Javascript

Javascript is an incredible language. The most incredible thing about javascript is how far it’s come since it was invented. Most languages follow a natural order of creation, obsolescence, and death. When weak programming languages don’t have what it takes they are surpassed by new languages and eventually die out.

Javascript is the exception to the rule.

Although Javascript was weak, being the technology that runs the web, it was not allowed to die. Instead, new features were taped, glued, and bound to the exterior with string and wire.

The Javascript we know, hate, and love today is a shanty town of features stacked on top of a core engine that was questionable from the very beginning.

The Problem with the Community

The real problem isn’t Javascript, it’s the way engineers are taught to think about Javascript. They’re taught that every language feature is special, useful, and has a valid time and place to be implemented so you should learn every feature and use them all whenever possible.

False.

Let’s take prototypal inheritance as an example. It’s weird, it’s ugly, and it has no place in a modern programming language. Just because it exists in the language doesn’t mean we should embrace it.

The Solution

I say no to all those filthy features. I simply refuse. Most notably, I do not use classes, the this keyword, decorators, or depend on any form of inheritance. All of my Javascript code is objects and functions.

I don’t do classes

A class is simply an object that has been initialized with state. No really, that's it!

The following two snippets are identical in spirit. The only difference, the class parser requires the new keyword while the const parser does not.

So why would I use a class? If you’re thinking about inheritance, consider the following snippet.

Anything you can do with a class I can do with an object and functions — and it’s not very difficult.

I don’t do decorators

I actually love the idea of decorators. Unfortunately, the Javascript committee was partying with the forbidden drink and found themselves drunk off their ash the night they decided on the format.

In simple terms, a decorator is a function that wraps another function, doing optional processing before and/or after the underlying function does its processing.

To accomplish this, Javascript created a format you have to learn. I prefer the Python approach. In Python, a decorator is a function that accepts a function as an argument and returns a function.

Here’s how I bring that to life in Javascript:

In this example, we’ve decorated the speak function with our uppercase and emphasize decorators to create the scream function. Now, in reality, we’re not decorating, we’re composing. That’s not a bug, it’s a feature. No matter, we’re accomplishing the same underlying goal of decoration.

But Why?

There are a handful of reasons why I limit myself to only objects and functions. Speaking in general terms, it helps keep my code simple, readable, fast, and just overall better. I like better.

Classes tend to Accumulate State

This is not a hard and fast rule but a tendency I’ve seen throughout my career. Personally, I strongly prefer deterministic code and — when possible — pure functions. If you're disciplined you can achieve these things with classes. However, it never seems to last. Even if I’m disciplined, it’s unlikely next 10 engineers share my sentiment.

Over time, classes tend to store more attributes and add more functions that depend on those attributes. When classes really get out of control the correct value of those attributes at any given time becomes crucial to the correct function of the methods. a.k.a the nightmare of state.

This type of stateful complexity is something I want to bring to the surface. Bringing state to the surface is a functional programming tenant. Stateful code and code that causes side-effects should be brought to the highest level of our software and everything below that top level should be strictly deterministic and — when possible — pure.

Using objects, I can write deterministic functions in the module scope and call them from my object instance, only passing what is necessary. This forces a manual passing of state to the functions and — hopefully — causes engineers to think twice about what they’re adding.

You could do this with classes but there are thirty-plus years of precedence for NOT doing it this way. When others dig into your code and find a class, they will increase the statefulness without a second thought.

Classes are Harder to Test

This point is tightly coupled to the previous one. The more state a class has, the harder it is to test it. When we remove the statefulness out of a class by making it an immutable object, then place complex functions in the module scope where they can be tested in isolation, we significantly reduce the complexity of testing.

I’m a big fan of code coverage for unit tests. Because of some painful experience, I can tell you that code coverage on stateful classes means NOTHING. On the other hand, if my classes are instead immutable object instances with deterministic functions in the module scope and I’ve covered 100% of my deterministic functions with unit tests, I can be confident the logic is actually covered.

Our motorcycle class is slightly absurd — everyone knows red motorcycles are the fastest — but it illustrates a real-world scenario where the state derives the result.

When writing tests, it might not be apparent that something as seemingly unrelated as the color of the object has an impact on the speed.

In the function & object version, writing tests for the getSpeed function would scream out to the developer that color — a required argument — has some effect on the result.

Objects Reduce Cognitive Load

As a human, in order to understand a class method, I have to load all the state into my own head and then consider the runtime behavior given the state it depends on. As developers, this usually means I’m scanning a class method for keywords like this or self to identify the dependent state.

Deterministic functions define their dependencies through their arguments. As a tendency, this makes them easier to understand and reason about.

Decorators Forever Bind the Decorator to the Function

When I use the @decorator feature in Javascript I am binding the decorator function to the decorated function forever. This completely breaks the ability to unit test my function in isolation.

Let’s look at a version of the speak/scream decorator example. This one has been converted to use the@decoartor feature.

I can’t test the speak method without executing the decorators. As someone who values quality unit tests, this is a non-starter.

Decorators Require Knowledge of an Obscure Format

Please look at the following snippet, our emphasize decorator, and tell me: what is a target? what is a name? and what is a descriptor?

The definition of a decorator is simple and beautiful — a function that takes a function and returns a function. Javascript defiled a truly elegant pattern with a convoluted format.

Decorators Require This

As seen in both of our decorators, they require the use of apply and this to correctly set up the context of the decorated method.

If you’ll just open your minds with me for a moment I’d like to share my take on this… it was a mistake 20 years ago and it’s a mistake today. Dear god please Javascript please stop building language features around it and engineers please dear god please stop writing code around it.

As an aside, if I’m interviewing you and I ask you to explain the this keyword in javascript, the correct answer is “I’m not sure, I don’t use that obsolete piece of trash, it only serves to needlessly complicate my code.”

I’m Ray, I’m a founding engineer at Vessel, I created Radash, the ACP design pattern, and the Exobase library. I don’t want your money, I just want to work on codebases that don’t suck so please follow, comment, interact, and let’s make better practices together.

TLDR;

If you’re writing ‘best practices’ applications using ‘modern design principles’ your code is probably a mess because you’re following a framework or pattern without thinking critically about what your code is made up of.

You Need a System

Before you can accurately implement a system of organization you first need the ability to identify what the elements you have to work with are. If you don’t know what you’re working with, how can you design a system around it?

Everything with a place, everything in it’s place

If a thing does not have an assigned place, how will it ever be put ‘away’ in an assigned place? My great uncle told me this in regards to lost toys when I was young. Lucky for us, it applies to life generally. A thing without a dedicated place of belonging can never be put in the correct place — it does not have one.

You’re reading this because you’re a nerd so with respect to software, what are the things in your application? Maybe some business logic? Have you identified it? Have you logically assigned it a place?

In my experience, you probably pulled a framework off the shelf and just write the code where they tell you to. Billion-dollar businesses have been built this way, but very few. Let’s do better.

The Types of Logic

It can be hard to distinguish one type of logic from another. We need to know what types of logic exist — in our code — and how to identify them. Otherwise, we can’t organize our application into a system correctly.

This is made worse when, having not been taught any better, we assume that there could be an unlimited number of unknown types, groups, and families of logic. This was me but it’s not true.

In any API there are only three types of logic: business, access, and utility.

Access Logic

Every external resource (i.e. anything that requires IO/async interaction — including the local filesystem) should have a module that owns its interface. This means that all interaction with an external resource occurs through a single module that has the responsibility of communicating with the resource.

The iconic example is the database. You should have a module that knows how to interact with the database and exposes functions that accept input without requiring any database knowledge.

A few examples of the wrong abstraction:

• Your database.js module exposes a function with the signature query(sql: string) => Row[]. This will cause higher level modules to know SQL, to write SQL, and to pass SQL down to the database module. Also, because the function returns the Row type (presumably a database row) modules have to know how data is stored so they can correctly parse it into a model type. We often store data in the database in obscure formats that are more preferrable to the database we’re using or more performant for our access patterns. Only one module should be responsible for knowing about that format, mapping to it, and mapping from it.
• You need to store files in AWS S3 so you make an s3.js module that exposes a function with the signature listInBucket(bucket: string) => S3Object . We can tell this is wrong because bucket and S3Object are both AWS S3 constructs. Higher level modules will have to know about buckets to pass the correct value. They also need to know what an S3Object is to parse the result into a file path or data stream they can use.
• You need to make http requests to a third party API, let’s say it’s the Abstract API, so you install superagent and make the calls wherever you need to. In your controller, in your endpoint, in your validation… anywhere.

Now, some correct abstractions:

• Your database.js module exposes a function with the signature listAllUsers() => User[] . As a new developer if I need to list all the users I can find the function and call it without needing to know anything about the database. A better design like this typically means a larger surface area on the exposed interface. Instead of exporting a single function query we’ll probably export dozens that are all specific to a model and use case. Internally, this database module will do mapping from the User model type to the type stored in the database.
• You rename your s3.js module to file-storage.js and export a function with the signature listInDirectory(directory) => File[] . Everybody, even the self taught engineers and those who went to a boot camp [caml down, I’m calling out myself] can understand this interface. Inside the module, we’ll map directory and file constructs to buckets and S3Objects.
• Abstract is a great example because they provide a dozen+ APIs. If your using their ip-geolocation API then you would make a module called geo.js and export a function with the signature lookupLocation(ip: string) => Coordinates . Without knowing anything about Abstract, anyone can see a function that takes an IP and returns map coordinates.

As a rule, if a module does not own the interface to an external resource it should be deterministic. This means any action should produce the same result given the same inputs. This is inherently not the case with a database where the same call executed twice could return different results (i.e. getting a user could return different results if the user just updated their profile).

These other modules are utility logic.

Utility Logic

If you have a file or folder called helpers, utility, shared, or common you are the problem and I am not speaking to you. Utility logic is not the catch all case for anything that you couldn’t find a good home for otherwise. Utility logic is the logic in any API written in any language and on any framework that is all the same.

If you’ve been working on backend services for long you’ve probably written query validation a few dozen times. Even if you used a library and let it do the work, you wired it up. Regardless of what your CEO says, that is not intellectual property. Nobody gives a sh** how you validate the query parameters sent to your API.

Routing, query param validation, json body validation, CORS, or anything else that has zero affiliation with your product, business, or idea— and does not own an external interface — is utility logic. If written well, the utility logic is the stuff you don’t have to change when you pivot to crypto.

You might be looking at your code and see that this type of utility logic has been wrapped up with all sorts of other logic and therefore is intellectual property… that’s bad, keep reading.

Based on the idea that this code doesn’t change between APIs you might think this includes modules like the database. Keep in mind that a good abstraction in the access logic surfaces functions that take in and produce model types. If you’re building a real estate platform, your database module takes in and returns models like House Property Buyer and Seller. This will change when you pivot to crypto so it is not utility logic.

Business Logic

Ah, the holy grail. It gives value to the business and purpose to the application. Simply put, it is the decision-making logic. The job of utility logic and access logic is to do what it’s told — by the business logic. Because of this, I often call it orchestration logic.

The utility logic handles a request, doing the typical work applicable to any API to prepare the request for handling, then passes the request to the business logic. The business logic orchestrates one or more access logic modules to get the data it needs to make a decision and use the access logic to persist or record that decision.

Business logic is all that really matters in a codebase. You may have 1,000,000 lines of code but only the 1,000 lines of business logic actually matter. Everything else only exists to support the execution of the business logic.

Business logic is valuable but also costly to maintain so we want as little of it as possible. We want dense business logic with high volume value and low volume code.

The Right Abstractions

The ability to distinguish one line of code from another is only helpful if you’re willing to use it to make the right abstractions. Depending on your chosen framework and patterns the abstractions can look very different. Here are some guidelines to use now that you can identify what’s what.

1. Being the only code that really matters in your application, the business logic should stand out at the highest level. When someone reads your code how many files, modules, classes, and lines of code do they need to read through before they hit the business logic? Good abstractions around business logic bring it to the surface and push the utility and access logic below.
2. Do not let access logic modules depend on each other. This is an anti-pattern I see often embraced as a feature and even mixed with business logic (I’m talking to you NestJS). If you need a dependency injection framework you’ve made a mistake. If the file-storage module needs the database module, stop to ask yourself why? Any case like this should be orchestrated with business logic. The business logic should know about both modules and call both of them, making decisions about how, why, and when.
3. Whenever possible, write utility logic in a way that abstracts the framework. Do not write a query validation function that accepts an AWS Lambda Context object as an argument. Similarly, do not write a JSON validation module that accepts the Express.js Request object as an argument. Utility logic is utility logic because of the fact it is applicable everywhere. Don’t limit yourself by writing it tightly coupled to a single framework.

The Moral

It’s possible you have a way you prefer to organize and abstract your modules. No matter how you use a pattern of abstraction like I’ve described here, or a different pattern that’s obviously incorrect, you can’t concretely apply any organization or abstraction until you can distinguish blocks and lines of code from one another.

I’m Ray, I created Radash, the ACP design pattern, and the Exobase library. I don’t want your money, I just want to work on codebases that don’t suck so please follow, comment, interact, and let’s make better practices together.

Better Practices

I’m going to propose solutions you’ve likely never heard of and write code you’ve probably never seen. You may feel the urge to retch and reach for the “best practices” that make you feel safe. I’d like to encourage you to keep an open mind.

The term “best practices” comes with an implication of time we don’t often consider. It’s better said in full “currently known best practices”. Times change and practices evolve. The practices evolve specifically to solve for the problems of the times.

Despite various publications of results where hand washing reduced mortality to below 1%, Semmelweis’s observations conflicted with the established scientific and medical opinions of the time and his ideas were rejected by the medical community.

- Wikipedia on Ignaz Semmelweis

Ignaz Semmelweis was the first doctor to connect hand washing to reduced infection rates. He evangelized his discovery to anyone who would listen. In return, the community — including his own friends — had him committed to an asylum. He was tortured and eventually died there.

Be Ignaz. Don’t be Ignaz's friends.

The Problem

The industry standard for backend development has, from time to time, made me consider quitting my job, leaving my home, and taking up a quiet quaint life of poverty. I’ve considered making a roomy two-man tent under a shady bridge in San Fransisco my new forever home. I might live in dirt and eat canned food but at least I wouldn’t have to write code that doesn’t scale, isn’t maintainable, and can’t be tested, changed, or understood.

It’s industry standard. These are “best practices”. Billion dollar businesses have been built like this. I’m so afraid of change…

[they respond in unison]

True, true, true, and ok… I can’t argue that the job isn’t currently getting done. I’m arguing that we can do better, a lot better.

Is good, good enough?

Testing

Mocking by creating an object that looks similar to another and passing it as an argument is acceptable. Using black-box tools like jest.mock to hack at a runtime’s import system and hot swap a real module for a fake one is not acceptable. I should not have to mock anything at the module/import level.

No matter the framework, language, or libraries, a testing suite should only require a slim test runner and, if needed, an assertion library. Testing should be easy. It should not require specialized tooling.

Martin Fowler provided a solution for this years ago: the Dependency Inversion principle. Since then, as engineers often do, a simple idea has been taken to the extreme with DI libraries, DI frameworks, injection containers, and dependency registration.

Keep that s*** away from me. I’m want to simplify my code and my tests, not the opposite. Using a DI framework to make testing easier is like trying to put out a house fire by loading it onto a truck, and dumping it into the nearest lake.

Simplicity is the way and there must be a way…

Behavior Isolation

I was once obsessed with the idea of business logic and the difference between it and other logic. What is business logic? Where does it live? How can I find it? If it’s the only thing that truly matters in every code bases why is it so hard to identify and find?

No matter the language, runtime, framework, or libraries every API is really just a lot of the same stuff. Endpoints validate requests, parse request arguments, make some decisions, access some data, and return a response.

The make some decisions part is where all the value is. That’s the business logic — or at least it should be. The problem is most code bases have this logic living intermingled with everything else. It’s difficult to identify or find. If it can’t be identified and found it can’t be protected and valued.

I should be able to open a code base and read the code, at the highest level, and the “business logic” should jump out at me. It should be like so totally obvi. As engineers, we write 1,000 lines of boilerplate to build up to the 10 lines of business logic that creates real value. Those 10 lines we worked so hard for, they should stand out.

Some say the M in MVC is the business logic. Bullsh**. I’ve worked on MVC projects in .Net, Node, Python, and Swift. You and I both know the logic is all over the place. Yes, I’ve read it in a book and I’ve seen it in a thirteen-year-old’s YouTube tutorial, but I haven’t seen MVC isolate and emphasize business logic in the wild.

If a solution requires a team of MIT grads to be used correctly I’m out…

Funnel Inversion

In most projects, there's usually some form of what I call layering. A request enters an application, travels down through the layers, eventually bubbling back to the top as a response. Your layers might be well organized but if logic for a single operation or use case is spread across them, is it easily understood?

We want our code to have a funnel-like shape. As I travel down through the application there should be less code and the code should be less specific. The code that’s important has been brought to the surface to create a broad top layer where the valuable code we care about is in plain view.

As a reader, to understand what an endpoint does in an inverted funnel codebase I have to traverse the code vertically, trying to first understand the child modules and functions before I can understand the top level. I should be able to skim one file to understand the behavior and intent of an endpoint. If I want to know how it works, of course, down I’ll go.

When making changes, the more code that is shared, the harder it is to make a change without unintentionally breaking other things. On the other hand, the more code that's repeated, the harder it is to make a change and be sure all needed updates have been made.

Some engineers worship the great god of DRY. I spit in his face. Contrary to the DRY ideology, there is a time to repeat yourself. I should be able to make a change to a shared piece of logic, but only for a single endpoint — easily and safely. This will require some repetition but with individual configurability.

The Only Escape is Death

As an engineer with a certain level of experience, you’ll come to know that there is no such thing as a correct decision. You can be right in two dimensions but in the third dimension — time — you’ll always, eventually, and inevitably be wrong. Because of this hard truth, I should be able to change my mind about the underlying runtime or framework without having to change my code.

I’ve worked on a dozen projects where we dreamed about changing the framework or infrastructure. Decision makers were always excited when we told them we could save thousands of dollars, 2x our velocity, and fix our worst bugs by switching from EC2/Express to Lambda (just an example). Understandably, all enthusiasm was lost when we told them we needed four engineers dedicated full time for three months to make the switch.

This is the nail in the coffin for the current “best practices”. Solving this problem is the differentiating factor for the Abstract & Compose Design Pattern. How much money is lost to startups and enterprises alike because they can’t make the changes they need to?

I should be able to write my code agnostic of the framework. I want to deploy to AWS Lambda, change my mind, deploy to AWS ECS, change my mind, deploy to Vercel, change my mind, deploy to GCP Cloud Run, and so on. Not because I’m manic, but because that's real life. Sadly, the industry has convinced us it’s not possible so don’t waste time talking about it.

It is possible, it has to be…

The Solution

To best understand the why and how we’ll take a snippet of common realistic API code and go through the steps to create a solution that solves for our stated problems.

A Quick Review

The problems we want to solve in our demo code:

• Tests should be easy to write. They should not require any module or file system mocking. They should not require a specific import order or a specialized testing library or framework.
• The business logic should be isolated, naturally emphasized, and set apart.

• Logic should be bubbled to the top so that the boring and repeated is pushed below and the important and special is at the surface in plain view.
• The implementation should not be dependent on, or have any knowledge of, the underlying runtime or framework.
• Boring utility code (like query string validation, JSON body validation, cors, etc.) should be modular and usable in any runtime or framework. Just because I move from AWS ECS running Express to AWS Lambda I should not have to change my code.

Genesis

In the beginning, there was an endpoint function with some logic for logging a user in. As an endpoint, its job is to validate the given credentials and return a token. Let’s assume it’s currently running on Express.

We’re skipping a few issues that would be really hard to step-by-step demo around module/logic organization. This endpoint is not layered, all the logic is already at the top. This means, there isn’t a function call like UserService.findOrThrow or AuthService.throwIfAuthMismatch where we would presumably push business logic into.

Step 1: Abstract the Framework

The first thing we need to do is remove the Express-specific (req, res) arguments and res.json function calls. We want a deterministic — or as close as you can get with a networked API — function that accepts the same input and returns a result — regardless of the underlying framework.

Not only will this help remove our strict framework dependency but it will make our code more testable.

We’ll do two things:

1. Create a generic Request interface that modules specific to any framework can conform to and all endpoint functions can accept as input.
2. Create a function that wraps ours, taking Express-specific arguments, converting them to our new generic interface, and applying the response of our function to the Express response.

Our generic request can be as simple or complicated as we need. If you think about the request object in different frameworks they’re typically very similar. They all have constructs like a body, query args, URL, IP address, host, and headers. As long as it’s not specific to a framework.

This function shape is one that we’ll see often so let’s name it… “hook”. When composing functions, all the functions we compose (aside from the endpoint), will conform to this hook shape.

In our useExpress hook, designed for the Express framework, the FrameworkArgs is (req: Request, res: Response) and the FrameworkResponse is actually void because the response is applied as a side-effect to the response object.

Using our generic Request interface and useExpress hook I’ll refactor the demo function to accept the Request type as an argument and return a result.

Excellent, we now have a login function with all our valuable code and it knows nothing of the framework.

We’re using the compose function from radash to do the composition. The compose function is super simple. If you don’t like Radash or don’t want the additional dependency you can implement it yourself in two lines.

Compose is just syntactic sugar. When you want to pass one function to the next, using compose you can write them in a much more readable series instead. If that doesn’t make sense, take a look at the following snippet.

If you’re familiar with Python this should be screaming decorators to you. They’re functions that take a function as an argument and return a function. If you have more of an OOP background just trust us… you’re in a safe place now, no one can hurt you here.

Dependency Inversion

By abstracting the framework we’ve also improved testability a bit. We can now call our function without needing to mock any global framework objects (like an express application object). However, our function is still referencing modules it wasn’t given as arguments. To test, this would require file-system/module mocking which is a stated problem to solve.

A bit of Dependency Inversion can go a long way here. I’m not going to install a DI framework, I’ll just extend the arguments we pass to our function and create another hook function to do the dependency injection.

Along with the req I’d like to pass services (the dependencies) now. I’m going to wrap them both up into an object I’ll call Props. A bit like React, this is everything our function gets passed. Because we’re doing function composition, it keeps things simpler to work with unary functions only.

Our function now gets an object with a services property that we can pull from. As a rule, I only put things into services that are non-deterministic. For example, the auth module’s compareCreds function is deterministic. Given the same input, it will always produce the same output (as any password comparison should). I won’t add it to services.

The current version of our demo code won’t actually work. We’re stating a Service type, and deconstructing services from the Props but we still need a hook to inject those service dependencies.

Here it is, our hook takes an object with the services you want to be injected into your endpoint function. This is a bit of a naive implementation. A more robust implementation accepts functions or promises as the values to the services argument and then makes sure they are resolved before calling the endpoint function.

If you’re into Abstract & Compose and you want to write your own hooks let me drop a bit of advice. Split your hook into a logic function — withServices above — and a wiring function — useService above. This makes testing your hook much easier!

I hate on DI frameworks but they do some things well. One of those things is ensuring an object gets all the services it depends on. With that said, DI frameworks are still trash because we’ve done that here with a few simple types.

Our endpoint states a Services type that it passes to the generic Props . Because of this, the endpoint can only access items in services that have been declared on the Services type. Our useServices hook also requires the Services type as a generic parameter and uses it to enforce that all attributes on Services are provided as an argument to the function. Typescript is now validating that our function has all the dependencies it needs to run.

Utility Abstraction

At this point, we’ve created a design I’d personally be happy to work on. We can still do better by taking advantage of the new hook concept. We have some utility logic mixed in with our valuable business logic. For the sake of isolating our business logic, removing what is boring and repetitive, and making common tasks more reusable, we’ll add another hook.

My goal is to remove and abstract away

• reading of the request body
• validating the body attributes
• erroring if anything is invalid

Before we dive into the solution I want to point out a few key concepts I’m using as a guide.

1. First, the data in the request is unsafe and untrusted until it has been validated and parsed.
2. The req should be treated as an immutable property so other hooks can depend on the original req always being accurate.
3. For the sake of clarity, our function should clearly define the arguments it requires.

Because of these three things I’m going to extend our Props type to include a generic args attribute. When we successfully validate and parse data from the request we’ll place the processed value in args without modifying req. Inside a function, for 99% of cases, you should only access request data through the args .

We’ll update our function to declare the arguments it requires as type Args and pass that type to the generic Props.

Our new hook will take the Args type parameter as well to help ensure that we are always validating all required arguments to a function.

Our new hook allows us to specify the schema and let the hook take care of the rest. The logic specific to the endpoint function stays in the endpoint function while the utility logic of validating and parsing is pushed below.

We can use this hook pattern to solve, in my experience, every common problem I’ve ever faced as an engineer. I’ve written hooks to handle CORS, caching, logging, alerting, metrics, billing, versioning, websocket messages, SQS record parsing, and more.

Bonus Features

There are some features you’ll get from using this pattern that we didn’t explicitly mention above.

Functional Infrastructure

Every endpoint is an entry point. By using compose to chain together the hooks and endpoint function we get a single self-contained function that has everything it needs to execute — accept the framework. This allows us to decouple the code from the infrastructure.

If every endpoint is a function in a file, the infrastructure is limitless. Our endpoints are like micro containers, able to be moved here and there without change. I can move a file that exports an Abstract & Compose endpoint function from AWS Lambda to GCP Cloud Run or Heroku without needing to make changes.

Language Agnostic

Although the initial library was written in Typescript, Abstract & Compose only needs functions to work. From my own experience I know I could implement Python, Swift, PHP, Java, Kotlin, Go, and C#.

An OOP engineer with a few beers and a slow Monday could probably find a way to make Abstract & Compose work via classes as well.

Performance

Endpoints using the Abstract & Compose design pattern tend to be faster than an equivalent endpoint not using the pattern. I’ll preface this by saying the amount of testing done has been minimal. However, it doesn’t take one of SpaceX’s rocket engineers to reason about why a composed function would be faster.

Let’s look at Express again, as an example. When a request enters it originates in the global application logic. That application routes it to the correct handler based on the path and method. As the request travels to the handler it goes through all globally registered middleware. The entire application is involved.

The opposite is true with composed function. Every endpoint function is composed of only the hooks it requires to execute. Assume a large project has 100 hooks and 100 endpoints. They have no effect on a new endpoint. The opposite is true with Express, if you have 100 middleware functions every request must be processed through all of them.

To be fair and maybe more clear, Abstract & Compose does not make your API faster. It isolates the factors that affect a function’s performance to only that function.

Exobase

I created a library, Exobase, that gives you common root hooks, auth hooks, request validation hooks, and the generic Props typing out of the box.

To be clear, as you saw above, no library is needed. The code in the Exobase library is rudimentary at best. You could probably have any junior engineer write up the wiring given good specs. As the creator and maintainer this a feature I hold dear.

Exobase provides a handful of packages you can use:

• The @exobase/core package provides the Props type, a functional structure for errors, and a utility for parsing results returned from endpoint functions. It’s three files. You could install core only and write your own hooks. See on GitHub.
• The @exobase/hooks package provides commonly used hooks to do CORS, query validation, json validation, header validation, and dependency inversion. See on GitHub.
• The @exobase/auth package provides hooks and models useful for wiring up your own authentication. See on GitHub.
• The @exobase/next package provides the NextJS root hook useNext that abstracts the NextJS framework. See on GitHub.
• The @exobase/lambda package provides the AWS Lambda root hook useLambda that abstracts the AWS Lambda framework. See on GitHub.
• The @exobase/express package provides the ExpressJS root hook useExpress that abstracts the ExpressJS framework. See on GitHub.

Real Life Code

I’ve written a few personal and contract projects using Exobase that are now open source on GitHub. You can jump in and see a few dozen Abstracted & Composed endpoints in the wild.

• Praxis endpoints
• IGT endpoints
• ShopSwap endpoints

Thanks for reading!

Even the best idea is nothing without a community — as Ignaz Semmelweis found out. If your a fan, please give it a clap, star it on GitHub, and share with a friend. If you have questions or comments I’d love to hear them.

- Ray

What’s wrong with Lodash?

In a time when Javascript’s dynamic abilities were held as a feature instead of a bug, Lodash was made to be as helpful as possible by providing functions that behave differently given different inputs. These days, we know better. Functional concepts like pure functions, deterministic behavior, and function composition have taken root in the javascript community — thank god 🙏

Lodash was created in 2009 (as Underscore) and rose to power after a fork (as Lodash) around 2012–2013. Do you remember Javascript circa 2012? At that time, I believed writing a backend in Javascript was irresponsible and foolish. The language was too unsafe, unmaintainable, and unwieldy. It’s a bit astonishing to think how far the language, runtimes, and community have all come in 10 years. It is not the same.

Lodash was designed and developed to solve Javascript developers' problems circa 2012. Those aren’t our problems anymore. The addition of optional chaining and null coalescing alone makes a large handful of Lodash functions obsolete. Not to mention the changes since the rise of Typescript. In today's market, a package as foundational as Lodash must provide helpful types.

A Function Breakdown

So you don’t think I’m just blowing smoke, I’ll walk through some of the Lodash functions and why they belong in a museum and not in your project.

The _.get function allows you to pass a string or array value to the path parameter. This was considered a dynamic feature back in the day but we should all know now it’s just a bug waiting to happen now. In Typescript the code _.get({ name: 'ray' }, 'age') won’t produce an error because Lodash’s types don’t verify the given string path exists in the object.

There are three issues here.

1. The function accepts different types as the second parameter. Again, no longer a feature.
2. When passing an object, array, or string the types don’t work. Because of Lodash’s poor typing, Typescript can’t check if the values passed exist on the types of the provided array.
3. Because of nullish coalescing and optional chaining, thefilter function is a bit obsolete. It was helpful when null checking needed many lines of code to do safely but any of the examples in the image above can now be written with ? and ?? just as same and just as short.

I won’t drag on these same topics but these points apply to many other functions as well. The _.find and _.some functions are just two examples.

Even more egregious are functions like _.map and _.size. They have all the issues mentioned above as well as the ability to accept a variety of values as the first and second parameters. Not a feature!

Code Quality

I could accept everything talked about so far but this is really where I draw the line. If I’m using a small helpful toolbelt library with functions I could easily implement myself — they’re not that hard to write— I should expect that I can easily look at the source code.

Here’s a link to the source code that the Lodash docs link to. Take a look.

There was a time when I was younger when I would have spent hours digging through the source, learning every internal API, cataloging every function call, and eventually understanding it well enough to answer a simple question like how does the isNumber function work?

I should not have to sift through 15,000 lines of code and learn internal APIs to understand how a one-liner function works. This is the anti-pattern of all anti-patterns.

I understand why this is the reality. If you are traditionally educated as an engineer you’re taught to be DRY above all else. If you’re not in the cult and haven’t drank the Kool-Aid you might know that there are things more valuable than DRY code. In some cases attributes like readability, maintainability, testability, and not pissing off your tech lead are only possible by repeating yourself.

Like anything, it’s a balance. Lodash, as a simple toolkit library, has gone off the deep end with its code base.

But that Community Though…

This is the one strength of Lodash. It’s old, everyone knows it, and StackOverflow is full of QA about it.

Also, to its credit, it was critically helpful to a previous generation.

What Then?

Radash, pronounced /raw-dash/ (Ramda + Lodash), is the Lodash of our generation.

• It’s a zero-dependency Javascript utility toolkit built in this decade.
• It’s written in Typescript so strong, and useful, types come prepackaged.
• It omits the functions that have become obsolete in Lodash over the years
• It provides new functions you’ve never seen before but always wanted (see below)
• The source code is maintained with understandability for newcomers as a top priority. In most cases, if you want to use a Radash function but don’t want to install it you should be able to copy it straight from GitHub.

The try function is probably my favorite Radash function. It’s the biggest hack to code cleanliness I’ve found in the last 10 years. No more forking the control flow when you want to try something. No more creating a mutable let variable outside the try, setting it inside, and checking it after.

As always with Radash, you’ll love the source and how strong the types are.

The parallel function has become my drop-in replacement for Promise.all . I know a team who has completely replaced the use of Bluebird in their project with parallel. Not only can you control how many requests are made at once but the error handling is far better. The parallel function throws an AggregateError with all child errors inside after all promises have resolved. This means the response from parallel isn’t an awkward array of objects with an error and value property, it’s just the values you want and expect.

Check out the source.

The retry function has become a replacement for the async-retry library for me and my team. Combine try, parallel, and retry and you have the tools to replace most backend service fault tolerance libraries out there.

Check out the source.

The counting function counts the occurrences of keys or identities in an array. It’s one of those small functions you find yourself using more than you expected.

Check out the source.

The range function is incredibly helpful when dealing with loops. When I see a raw for (let i; i < n; i++) loop I tend to throw up in my mouth a little. The range makes the same kind of function so easy and simple that you can get the same behavior without of the smelly code and risk.

Range returns a generator so you can safely call range(0, 100000000)

See the source.

The list function is the range function’s beautiful cousin. In some cases, you need a real list, not a generator. In those cases, you have the list function. Easy, quick, and dynamic list creation is also useful in a lot of use cases outside of looping.

See the source.

There are so many more amazing functions. Here are a few more of my favorites.

• objectify and listify
• throttle and debounce
• pick and omit
• compose and chain
• memo and proxied

All functions are well-typed, well-documented, well-tested, and written with simplicity as a priority. If you find that I’m wrong please submit a PR or open an issue!

The one thing Radash is missing that Lodash has is the community. If you like Radash, share this article, star it on GitHub or share it with a friend.

I wrote and maintain Radash. This changes nothing.
- Ray

Lodash is dead. Long live Radash. was originally published in Exobase on Medium, where people are continuing the conversation by highlighting and responding to this story.

But how do you get it? If it can’t be measured, tracked, or valued concretely then we’re just making it up. I’ll concede it’s a bit ethereal but it’s not arbitrary. After 11 years in the industry, working with other engineers of all levels, I‘ll tell you what I see: Your engineering level is the delta between your knowledge and your experience.

Knowledge & Experience

Let me define these in terms local to the context. Your knowledge and your experience are opposite sides of the same coin. They are the same statement made once in a postivie and the other in a negative. Knowledge means you know how to do a thing. Experience is to know how NOT to do a thing. You can gain knowledge through a variety of educational methods, it’s information you learn. You can only gain experience by doing something wrong, by making a mistake, or by failing.

Some extraordinary people have the ability, to a higher degree than others, to learn from other people's mistakes. This skill is not common to engineers but it’s a superpower if you have it.

On your first official day working as a software engineer you have zero experience and an overflowing amount of knowledge. That knowledge may have come from university courses or — as in my case — YouTube. You are a Junior Engineer on this day because you have too much knowledge relative to your experience. That's right, I said it like it's a problem because it is.

Junior Engineers believe that they’re right. They’re wrong, they just don’t know it yet. They don’t know it because they don’t have the experience. When you have experience (again, mistakes and failures) it will hit you over the head reminding you it’s possible you could be wrong. Junior Engineers, having an abundance of knowledge and a total lacking of experience, often don’t even consider the possibility they’re way off course.

I’m not bashing my juniors out there, I was there once. I remember so clearly standing in a small meeting room, around my third year as an engineer, marker in hand, waving furiously at the whiteboard, and arguing as though my life depended on it. I was convinced, beyond reason (as I can only see now), that users MUST have the ability to choose their own username and that the system would fall apart if they were forced to use their email to log in.

Looking back now, it was so trivial…

That Senior Engineer, Dave, let me have my way and I was wrong. For our use case, there were many pitfalls with the username system. Dave saw them coming, he tried to warn me, but I wouldn’t listen. As we worked through the project Dave was kind enough to point this out to me.

Although I regret the decision, I’m thankful because it provided valuable experience. I now understand the pitfalls of a username-based account system again. I did it wrong. That’s experience.

You might notice if you're a critical thinker, that a username-based account system isn’t necessarily wrong. I know. I’m not saying I’m correct in my feelings. The value of experience doesn’t come from how it changes your opinions, it comes from how it teaches you that you could be wrong. Nobody is a senior engineer because they have all the correct opinions.

Many times over my career, when having an argument, I thought back to that whiteboard with Dave and how I believed beyond reason that I was correct. It makes me question what I currently believe and consider the possibility that I’m wrong.

Having more Experience

Nobody wants to be a mid, being lost in the void between junior and senior. It’s a second-place price reminding you you don’t have what it takes to make senior. Once you have enough experience to question your knowledge you are officially a Mid Level Engineer. This makes it a genuinely important step and stage.

You’ll know you’re there when you think you know the answer but you look up alternate solutions anyway. When you’re in an argument you won’t cling to your opinions like they provide the oxygen you require to take breath.

More Experience than Knowledge

You’re a Senior Engineer when your experience surpasses your knowledge. At this point, you’re more likely to recognize a faulty belief than not. In the job that you do, you have more lessons sourced from mistakes and failures than you do from textbooks and blog posts.

There is an important assumption I’m making here that's important to recognize. The technology industry contains near limitless information available to be learned and absorbed as knowledge. Your knowledge and experience are vertical restricted, they don’t apply to the entire industry.

I’m a fullstack web developer sort of engineer. I’m familiar with clouds, API design, frontend javascript engineering, and other stuff you’d expect from a web dev. Around these topics, I’ve built up a lot of knowledge and experience. If I decided to join an embedded programming team at SpaceX, writing the C code that's loaded into the microprocessor on a servo that controls the pitch of a rocket's arms… my wealth of experience would be worth nothing.

The point, senior engineers are senior on particular subject matters. If you’re young and wondering if you should go deep in one area or go broad in many keep this in mind. It’s possible to be a true fullstack engineer and go deep in both front and backend for example but as I can tell you, it might take up most of your twenties.

dFor the last few years, I’ve maintained that there has to be a title above Senior Engineer. It simply doesn’t make sense, based on my experience, that Senior Engineer is the ceiling for an individual contributor — or anyone else who works hands-on in the code. I’ve seen engineers who operate at a whole different level and recently I can say I’ve joined them.

I’ve always kept my hands deep in the code but over the last four years, I’ve been a tech lead (in different capacities and titles) leading senior engineers. What I’ve found is that it’s impossible to be completely right. You’re always wrong, if not now, then sometime in the future.

The problem is that businesses change, customers’ needs change, markets change, products pivot, engineering teams and their expertise change, and so on. All this means that what we build as engineers will eventually, one day, have to change.

This is another level of realization that requires a lot of experience over time to see. Choosing AWS Lambda instead of GCP Cloud Run as infrastructure won’t be the right choice if AWS goes bankrupt in ten years. There are things that are out of your control that you can’t predict.

As a final-level engineer, you identify the things you can control and predict and build your software around them. Everything else is an externality that your system should be abstracted from to allow for quick and easy change.

How do you abstract away elements as deeply connected and ingrained in your application as the underlying infrastructure of framework? That’s for another article.

Relativism: What’s the point?

A note about being wrong and right. Sometimes you’re wrong… but sometimes you’re right. Not all ideas, designs, solutions, or implementations are equal. Some solutions are better than others.

Knowledge is the gas pedal, driving you forward in an argument. You have facts, code samples, articles, best practices, and a whole chapter from a Martin Fowler book on your side. These things fuel the thought, the instinct, that you’re right. They drive the pedal down and the argument forward.

Experience is the break, a lifeline to slow and stop the argument. Memories of previous mistakes and a healthy fear that your solution won’t actually work bring clarity and make an opening for you to consider an alternate viewpoint.

If you can’t intelligently argue for both sides of an issue, you don’t understand the issue well enough to argue for either.
- GBlink’s Dad

Sometimes you’re right, and when you are, push. The team, company, and project depend on the correct decisions being made. Sometimes you’re wrong, and when you are you won’t be able to see it if all you have is knowledge driving you forward into an argument.

How do you concretely evaluate someone’s knowledge and experience? I’m not sure yet. I’m working on it. For now, I just know this is the way.

You’re Not a Senior Software Engineer was originally published in Exobase on Medium, where people are continuing the conversation by highlighting and responding to this story.

As a small child, one who tended to loose toys and jackets, I was thought this phrase:

Everything with a place and everything in its place

The idea: if everything has been assigned a location then everything can be put where it belongs. If an item does not have an assigned location it can’t be put away. A terrific lesson for children and engineers alike.

As engineers, we should think critically about our types, classes, layers, services, and the other devices we use to organize our software. Everything should have a dedicated purpose and home where it belongs. After 10+ years of working on APIs in disarray, this is how I organize my types.

The Only Three Types

There are only three types in an API. They are the view, model, and storage types. Of course you may have others but they should be isolated and abstracted into modules where the interface interacts strictly by model types.

As a request comes in we validate and parse it to produce arguments, often partial model types. When putting model objects into the database we convert them to storage types. When pulling from the database we map the storage type to a model type. When returning models to the client we transform them into view types.

Model Types

This is what you’re probably used to. Everything in your API is probably a model type right now. Model types are the shapes that the API expects to work with. They are the standard shape of data. Let’s assume your API is a public community commerce where users can post listings they’d like to sell.

Storage Types

When you store a model you often modify it to enrich the object with properties that are helpful for the specific storage your using. For example, when using Mongo I take any foreign key properties and add a new property where this id is converted to Mongo’s ObjectId type. This makes Mongo’s indexes and queries faster.

In a SQL database, you might want to convert millisecond timestamps from your model into Date objects or flatten properties represented in the model as arrays or objects.

We call these storage types instead of database types because in modern applications we’re often storing different data in a variety of storage devices. Your storage types might cover timeseries data you put into Mongo, relational data you put in Postgres, payload data you put in SQS, event data you put in Dynamo, and cached data you put in Redis. Storage types cover all these modules. Each of the modules should accept and return model types.

View Types

These are the types surfaced by your API. Often times your model includes data you don’t want exposed to the client. Or maybe, you only want it exposed in different endpoints for specific authorization levels. View types clearly express the data that should be returned and if done properly will enforce correctness and disallow over-exposing data to the user.

There are only two differences between this type and our `Listing` model type.

1. We’ve added a _view property. This is how we ensure that the `Listing` model can’t be unintentionally used in place of the `ListingView`. Because `ListingView` is really just a subset of properties from the `Listing` model, Typescript would allow the `Listing` model to be used if not for a unique property on the view type: `_view`.
2. We’ve omitted the `reported` property. For our use case in the basic find listing endpoint we don’t want to expose how many times a listing has been reported.

Storage Mappers

To maintain this abstraction, any storage module needs to accept and return model types. However, internally, those model types should be mapped to storage types when storing and mapped back to model types when reading from storage.

This should live inside any database or storage module you have. This allows the business logic to focus on business logic things using the model types.

View Mappers

A small module with view mapping function handles converting the model type to the view type strictly and safely.

Endpoints

It all works together in an endpoint. A request enters, model types are created, storage modules are called with model types, they return model types, any business logic is done, and the endpoint returns a view type.

The Only Three Types in an API was originally published in Exobase on Medium, where people are continuing the conversation by highlighting and responding to this story.