You’re one week into your programming course, and it feels like you’ve already hit a skill ceiling. You’ve checked every line, you’ve been staring at the screen for an hour, and yes you’ve checked the damned semicolons. As if to mock you, the computer’s gone ahead and put a warning on the same line as the error, so you can’t even click both of them at the same time.

You give up trying to fix this yourself, and ask for help. Your friend looks at your code for two seconds and points her finger at a particular spot.

“There needs to be a closing bracket right here. Right now this else is inside the if.”

Of course. You just learnt about if-else statements. You’re supposed to know this stuff. You feel like an idiot. If I can’t even get the brackets right, maybe I’m not cut out to be a developer.

Frustrated, you reach over to close your laptop.

Stop.

It’s easy to blame yourself when you make very basic mistakes, but this is something every programmer goes through.

Like any other craft, programming has skill levels. However, programming is unique in how unforgiving it can be when you mess up.

Say you’re learning to paint. Learning to use the brush is a lower level skill than composition. But, once you get to a reasonable level of skill with the brush, you can start working on composition. Even if you mess up your brush technique a bit, it doesn’t stop you from practicing higher level skills.

This is not how programming works, especially at the lower levels. You miss a bracket, you’re out. It doesn’t matter if you understood the logic, or you have a brilliant solution to the problem. Your program just stops dead in its tracks.

This is the beginner programmer’s curse: to become better, you need to practice writing code; but if you write imperfect code it may not even work.

So, what do we do? Well, getting good at any programming concept involves 3 different skills. You need to develop all three, and often what’s holding you back is that you haven’t paid enough attention to one of them.

We’re going to break down those 3 skills that you need to build to break through that skill ceiling you’re hitting.

Skill #1: Knowledge

This is the obvious one. You need to know and understand the concept you’re learning.

What’s an if statement? What’s a function? What’s a loop?

You’re smart, and you got this. Trouble is, programming is a skill, not a repository of knowledge.

Maybe you know what a butterfly stroke is. Maybe you’ve even watched the butterfly stroke event in the Olympics. But if that’s all you’ve got, I don’t think you should sign up for life guard duty at the local pool just yet.

You need to go beyond just knowing and understanding. There are two more skills you need to develop:

Skill #2: Problem Solving

It’s one thing to know what an if statement is. Knowing when it needs to be used is a whole other skill.

If you’re having trouble figuring out what to do next, you have a problem with application.

Application is about knowing which tool in your belt you want to reach for to fix this specific problem.

The only way to get better at this is practice. Specifically, you want to try many different problems that test the same concept. Try to find a resource that categorizes practice problems by concept (variables, conditionals, etc) for your language.

Don’t stop after you solve the first and easiest problem. The point isn’t to prove that you know what an if statement is. The point is learning to recognize the type of problem that an if statement solves. By doing multiple problems that use the same concept in different contexts, you learn to see the shape of problem that this concept can help you solve.

When you move on to trying problems for a new concept, your brain will instantly spot all the bits that can be solved using lower level concepts. When picking apart the problem like this, you will much more easily spot the parts of the problem that require the new concept.

To get better at application, practice with multiple problems that target the same topic.

Skill #3: Muscle Memory

Programming is a craft. It involves creativity and logic, yes, but it also involves using your eyes and fingers to perform tasks. It’s not enough to know what needs to be done, you need to be able to actually do it effectively.

You see this in other crafts as well. You might know the notes of the song you want to play on the guitar. Maybe you’re even a trained musician in a different instrument, and your brain is coming up with improvised melodies. But until your fingers know how to listen to your brain, you will not be a guitarist.

This ever happen to you? You come up with an idea: I need to check if the password string’s length is too short, and then pull up the error message if it is.

You’re pumped to write the code, but you stop short: How do I get the string out of that text box, again? Wait, is there a round bracket after the “if”?

These tiny questions are easily solvable, but together they hold you back from solving the bigger problem. It becomes frustrating for you to progress to more complex problems, because your brain keeps having to deal with these smaller issues.

If you’re having trouble actually writing the code, you have a problem with execution.

How do musicians train execution? They do drills. As a programmer, you want to do something similar.

Take problems you’ve already solved, and do them again. This time, there is no logic puzzle. You already know what the solution is. You’re simply training your eyes and fingers, and building muscle memory.

Doing drills trains you to very quickly spit out the code for a given idea. Want to compare a string’s length? if, open bracket close bracket, open curly close curly, move cursor back inside bracket. Boom. Now you can think about what the conditional should be, instead of how to write one.

Doing drills also trains your eyes to spot errors more quickly. You get more accustomed to what correct code looks like. So when you miss a semicolon, or a code block is wrongly indented, it jumps out at you immediately.

Drills make you faster at both coding and debugging.

As you do this for each new concept, more and more skills move to this subconscious level. Soon, your brain will be planning larger blocks of logic while your fingers spit out the relevant code.

Initially, simple concepts enter your muscle memory: brackets, semicolons, variable declarations. Soon, things like function definitions, class definitions. Then larger tasks, like adding a new screen to your app and wiring up the relevant code.

You can hold larger and larger problems in your head because your brain does not need to be distracted by the actual execution.

To get better at execution, practice the same problems repeatedly over time.

Know what to practice

When you’re stuck with a programming concept it can feel like you’ve hit a skill ceiling as a programmer. Practice is really the only way to progress past that. But it’s important to know where to focus your practice.

When you get stuck with a concept, use the guide above to try and identify which of these 3 fundamental skills is holding you back right now.

Focus on getting better at that skill, and you’ll smash through your skill ceiling in no time.

If you found this article useful, you can help me out by sharing it! You can also subscribe to my mailing list to be sure not to miss my next article.

The Beginner Programmer’s Curse: Why learning to code is frustrating was originally published in HackerNoon.com on Medium, where people are continuing the conversation by highlighting and responding to this story.

The greatest learning system we know about is the human brain. It’s made of billions of really simple cells called neurons. Our intelligence arises from the complex connections between these neurons.

Neural networks are modeled after this.

Each layer of neurons learns something small and passes it onto the next layer. Layer by layer, the network can learn very complex things.

The exact way we connect and arrange these layers creates networks that are great at different types of problems. Here’s how a network might recognize objects in an image:

The first layer simply learns to recognize pixel brightness. The next layer learns to combine them into line segments. Each subsequent layer learns increasingly complex visual features. By the last layer, the network has learnt to identify objects of interest.

Neural networks are an amazing tool with many applications, from computer vision to natural language processing.

Neural networks: The 1 minute guide was originally published in HackerNoon.com on Medium, where people are continuing the conversation by highlighting and responding to this story.

And then you hear it. A fragment of conversation floats over to your ears:

“…they’re definitely the machine learning startup to watch.”

No…

“Don’t they still use statistical NLP though? Recurrent neural nets seem like the way forward…”

Not again…

And like a virus, it spreads. You watch in horror as, within minutes, everyone at the bar is talking about machine learning.

You’ve heard the term before, of course. Who hasn’t? Machine learning is the way forward, the future of artificial intelligence. Of course it is.

Trouble is, you don’t have a damned idea what any of that means.

You down your drink. It’s going to be a long night.

The end of the world

First, we hear that machine learning bots are taking our blue collar jobs.

Then, we find out white collar jobs aren’t safe either.

Resigned to our jobless future, we find out that the bots are now talking behind our back.

Gossiping about us lazy unemployed humans, probably.

Machine learning is talked about in so many different contexts that it can be hard to grasp what exactly it is. You look it up and get abstract theoretical explanations, high power Scrabble words, and a wall of math and code.

You just want to know what exactly machine learning is, why it’s a big deal, and maybe a bit about how it works. Honestly, you just want to make sense of all the buzzwords that get thrown around, like curse words in a middle school classroom: everyone’s using them, but you have a feeling nobody else knows what they mean either.

Let’s dive in

So, machine learning. Is that like… AI?

That’s a good place to start. We all know what artificial intelligence means. I mean, we’re all thinking of different things, but surely it’s the thought that counts.

Artificial intelligence (AI) is the study of building systems that can make “intelligent” decisions.

Basically, if a computer does something that seems somewhat smart, we label it artificial intelligence.

Let’s use an example you might have come across. Computer games often feature enemy characters who appear intelligent. They follow us around, and they act in ways that make the game a challenge. That’s an example of AI.

The developer of the game has achieved this by giving the AI a set of rules. Follow the player. If the player is shooting, find cover. If the player stops shooting, try to shoot the player. The more of these rules there are, the more intelligent the game appears to be.

Thing is, a computer game is usually pretty limited. The player can only perform a few specific actions, and the level was entirely designed by the developer. So, developers can come up with rules for great AI characters that seem really intelligent.

Not all problems can be solved with rules

Say we want a computer to detect if this is a picture of a dog.

How would we even start defining rules for this task?

No, really. Try to think of the kinds of rules we need.

Dogs have four legs?

Dogs are white in color?

Dogs have fur?

Computer vision problems, like recognizing an object, are really complex. But our brain solves them almost intuitively. So, it’s really hard for us to come up with explicit rules.

Enter machine learning

We don’t build a system that recognizes dogs. We build a system that can learn to recognize dogs.

We feed it thousands of images, some containing dogs and some not. The system learns the rules that define a picture of a dog.

Machine learning is the study of building systems that can learn to make intelligent decisions.

The keyword “learn” is what distinguishes machine learning from other types of artificial intelligence.

“You said the machine can learn the rules. So clearly, the rules do exist. We’re just not trying hard enough to find them.”

You’re absolutely right. If we tried hard enough we can probably develop a rule-based dog detection masterpiece. It would have thousands of rules covering every possible scenario, and it would be a thing of beauty.

What if we now want the system to detect pictures containing food? Well, we throw the whole thing away and start from scratch. None of these rules would apply.

A system that can learn to recognize objects can be trained to recognize food simply by using pictures of food instead of dogs.

That’s the power of machine learning. Instead of a fixed set of rules, we have a flexible machine that can learn and adapt.

Keeping it real

Now, I totally made it sound like we’ve built a living, learning being and taught it to recognize dogs by letting it play with puppies. Then we teach it the meaning of cuteness and the meaning of love, and hopefully convince it to not rise up and kill us all. Right?

The reality is a lot less complicated. We want the machine to learn to make decisions, usually based on predictions. So let’s start by asking this: how do humans make predictions?

Here’s an example of how we make predictions. You might have even done something like this in Science class in middle school.

Say you conduct an experiment. You collect data, and plot it on a graph.

The x-axis is the value you changed, and the y-axis is the result you measured. You then draw a trend line (or best-fit line) on the graph.

This is the line that best models the data.

Now that you have this line, you can make predictions. You can pick a weight that you’ve never measured before, look it up on the x-axis, and read off the y axis to predict what the spring’s length will be.

You used data, built a model, and used it to make a prediction.

If you wrote a program that can perform these steps, congratulations: that’s machine learning.

No, really. Finding the linear model and using it to make predictions: that’s totally a machine learning technique, called linear regression.

Couldn’t we just program the model into the system? The system could then make predictions about this spring. But it can’t make predictions about other springs without us writing new code.

If, instead, we teach the program to perform linear regression itself and learn the model, that’s machine learning.

We don’t provide the model. We let the machine learn the model.

Hocus Pocus

If there’s one thing you take away from this example, it’s this:

Machine learning isn’t magic.

We’re not building a self aware being that can learn everything. The spring model program will never learn to recognize dogs.

We built a system that can learn one particular type of model. All we’ve left out is what the model is.

Almost every machine learning system does some version of the following:

The system gives us an output based on the model. It evaluates that output to determine how bad it was. It then updates the model to make future outputs slightly better.

Machine learning is the art of building a system that can perform this loop for a specific problem.

The output can be wildly different depending on the problem. It might be a text response from a chatbot. Or it might be a decision on whether or not to hit the brakes in a self-driving car.

The actual model could be linear like in our spring problem. It could be a more complex model based on statistics and probability. Or, it could be a neural network, designed to emulate the way the human brain works:

Neural networks: The 1 minute guide

Whatever the details are, almost every machine learning system is running that loop. With each cycle, the system improves a tiny bit. After thousands or even millions of cycles, we can get systems that have learned to perform better at that task than any human.

The sheer variety of applications, and the techniques needed to solve them, is why machine learning intersects with so many other fields. It’s also why machine learning is such an exciting topic right now.

There we go

Building a machine learning system can be complicated, but understanding machine learning doesn’t have to be.

So, go conquer that startup meetup. Throw out a few of those newfound buzzwords. Network the heck out of that room. You just need to do it for a few more years, really. By then, I’m sure we’ll have bots who can do that for you, too.

If you found that interesting, check out my 1 minute guide to neural networks!

Subscribe to be notified about my next article:

This story is published in The Startup, where 260,100+ people come together to read Medium’s leading stories on entrepreneurship.

Subscribe to receive our top stories here.

The non-techie’s guide to machine learning was originally published in The Startup on Medium, where people are continuing the conversation by highlighting and responding to this story.

However, to launch on another platform you need to start over, using that platform’s toolkit.

An alternative is a web app — a web site that offers rich user interaction, just like an app. Built using web technologies like HTML, web apps are accessed using a web browser. You can reach every platform by building just one app!

The downside: web apps can’t access all of a platform’s functionality, like using Bluetooth or running in the background.

Meet their love child: the hybrid app.

This is a web app hiding inside of a native wrapper — basically a tiny browser that only ever shows one web app.

Hybrid frameworks like Cordova even offer toolkits that let hybrid apps access many native platform functions.

You write the code just once, but the resulting app can be distributed using the app store on multiple platforms.

Hybrid Apps: Explained in a minute. was originally published in HackerNoon.com on Medium, where people are continuing the conversation by highlighting and responding to this story.

There’s a difference between the two forms of communication. The written word is visual, whereas the spoken word is auditory. That sounds obvious, but here’s what’s important about that difference:

Speech is intrinsically tied to time: you hear it one word at a time.

Written text is a bunch of symbols that are all already on the page. It’s possible to write and consume it out of sequence.

It’s easy to write a great paragraph that has no link to what came before or what follows.

This is something you just can’t do with speech. When speaking, you have to start with the setup, and then take the listener through each beat until you reach the conclusion.

Speaking forces you to consider the narrative momentum of your arguments.

When writing, it’s very easy to write an article that is just a collection of points. When speaking out loud, you instantly feel awkward when you switch to an unrelated point without the requisite transition.

How to use speaking

Speaking can be used as a tool to develop your article. It can also be used to refine and edit what you’ve already written. In either case, here’s what you do:

1. Step away from the desk. You want to use the speaking part of your brain, so put away your notes or text.
2. Start at the top of the section you’re working on, and talk through your points. Think of this as a lesson or speech that you’re giving. When you get stuck, you can stop to think or consult your notes. The start at the top and go again.
3. Do not take notes as you speak. Unless you have a spark of brilliance that you must absolutely preserve on paper, stay away from any note taking. Iterate on the text entirely orally. Use your memory as a quality filter: if you are forgetting something, chances are it doesn’t belong there. In a cohesive argument, every sentence shows up exactly when needed. If you’re forgetting lines, it might be because you’re delivering a bunch of unrelated points. Try to find a better order and try again.
4. 1–2 minute is probably all you can hold in your head without memorization. At that point, come back to your desk and start writing.

Things to watch for

Speech has some distinct characteristics that don’t translate well to the page:

• When speaking, we are prone to repeating ourselves. On the written page, needless repetition stands out.
• Speech is often filled with very long sentences that can still be understood because of tone and pacing. Writing is visual, and a wall of text looks intimidating.

When writing, remember that you are not transcribing a speech. You are adapting it for text. You will need to break up sentences. You will need to use punctuation and whitespace to bring out the article’s structure. Speaking helps you with structure and flow, but can produce poorer sentences. Bring out your inner writer and editor to fix those problems.

The point is not that speaking is better than writing. It’s about forcing the article through the speech, writing, and editing parts of your brain. Each brings a different set of competencies to the table, and all of that benefits your article.

You may not think you’re a great writer. Maybe you don’t think you’re a great speaker or editor either. But by involving all three, you get to cheat. You’re rigging the game, and bringing three of you to the fight.

Want to write better? Step away from the keyboard, and speak.

To make sure you don’t miss future writing tips, subscribe to my newsletter!

This story is published in The Startup, Medium’s publication followed by over +256,410 people.

Subscribe to get the top stories here.

Want to write better? Step away from the keyboard, and speak. was originally published in The Startup on Medium, where people are continuing the conversation by highlighting and responding to this story.

It’s as if the project has a life of its own, and is trying to destroy your life.

How did this happen? Did you hire a bad developer? Did you fumble the project management? Or was the idea just terrible all along?

Possible. But there is one core misunderstanding that often dooms a project from the start.

We assume that our product is defined by a set of features, written on a sheet of paper. Occasionally features get added; occasionally features get cut. But you can see the scope of the project at a glance.

This assumption is wrong.

Your project is not two dimensional, like a sheet of paper. It has depth.

Each feature can be peeled back to reveal layer after layer. In fact, if I were inclined towards creating clickbait titles, I’d say that your app is an onion. Let me show you what I mean. Let me tell you why, as you peel each layer back, your app makes you cry.

The library to end all libraries

Let’s look at something concrete. Pitch me an idea.

People often want books that nearby libraries don’t have. Other people in the area might. So this app lets people send out book requests, and people who have the book will respond. We make money off each transaction.

Genius.

I know, right? The feature list is straight forward:

Time to send the spec off to the developer and start thinking of killer startup names. kaBooki? lib.rari.ly? reddit?

Alright, hang on. Let’s take a closer look. Also, reddit is already a thing.

Let’s probe the idea with some questions and see where it takes us.

How do users pay each other? Do they just chuck cash at each other when they meet, or do they pay through the app?

Oh, they should pay by credit card through the app, so we can take a cut.

How do they actually request a book? I mean, what do they literally do on the app? Do they fill up an open-ended form with the book title and author?

Oh, hmm. They should search for their book and pick the one they want.

Okay, so we need a database of books.

I guess.

Actually, what is a book request? Do book sellers see a list of all requested books? Or do we send out notifications to sellers, like Uber or Thumbtack do?

That one. We send out requests to other people who have the book.

So, sellers need to populate the app with all the books they are selling?

Um, yes.

How does the book actually get across? Do users send it themselves, or do we handle the delivery?

We handle the delivery.

So you’ll need a system that tells you the books that need picking up and where they need to go. I imagine your drivers will want to see the shortest route that connects these, right?

Okay, no. How about this? Users mail the books to each other.

So we show the recipient's address to the sender? How do we handle shipping charges?

Actually, you know what. These users have to be near each other. They can just go pick it up, and they can make a new friend, okay?

So, we need to match users who are close by. How do they find each other? Does the app have a real-time map so they can arrange a meet-up?

Sure. Fine. Whatever.

Do we set the book’s price? Or do they negotiate the price themselves?

They talk it out and decide on a price.

How do they decide? Do we need to build chat into the app?

They can just call each other on the phone.

Alright, so do we want to verify that number with a SMS text message? What about… you get the point. We really could go on forever.

Please don’t.

Let’s have a look at what the feature list looks like now:

Every single one of those line items can be expanded multiple times. We just peeled the onion back one layer, and look what happened.

Now now, don’t cry.

What happened?

Let’s be clear about what the reason for the feature explosion is not.

This is not what is commonly known as feature creep.

That’s when new functionality keeps getting added to a product. If you look at our final list of features, each item exists to support the original pitch.

This was not because of technical considerations.

For example, what is the most suitable chat system architecture? What payment platform is easiest to integrate?

We are not talking about those here. A good developer can talk you through various technical options. But a developer can’t decide for you what features need to be built.

This was also not because of market considerations.

Do people want to use such a product? What incentivizes early users to sign on when there are few other users? Are users willing to pay for this service?

These questions have to do with validating your idea, identifying product-market fit, and pricing strategy. Again, these are important issues. But this is not why your feature list blew up.

Yes and it’s also not because of global warming, or the tooth fairy. Are you going to tell me the reason for the onion, or go through every single non-reason?

Oh, you’re no fun. Here’s what happened.

We fundamentally misunderstood how complexity works.

We thought each feature is as complex as the description we started with. Hey, maybe it takes time to write code or whatever it is engineers do, but the feature can be fully described with a few words framed in business terms. “User does x with y”. Right?

This view is wrong.

Software features are more like fractals

The closer you look, the more details make themselves apparent.

Onions are fractals?

No, I’ve switched metaphors. Keep up.

We started with a big picture description of what the feature is. However, when the app is built (by the developer) or used (by users or yourself), it is at a much smaller scale. It is done by going through the app step by step. At each step, you might encounter problems. To solve the problem, you may need to redesign that step, add new steps, or add entirely new features.

It’s like planning a journey from your home to your office. Looking at the map, the journey might look like a straight line.

When you get down to a smaller scale, you’ll see that your straight line doesn’t fit in the city grid. You encounter traffic stops, and elevation, and obstructions. Depending on whether the directions are for walking, biking, or driving, you need different routes.

Building software is much the same. The closer you look, the more detailed you realize your solution is going to have to be.

Great. Details all the way down. Just what I needed. Does this mean my product will never be finished?

Well, yes and no. The complexity showed up because we started with a feature description, which is usually an objective. We then zoom in and build every step of the solution to achieve that objective. The more we zoom in, the more complexity we will uncover.

The complexity isn’t the problem, though. The problem is the way we choose to uncover it. We hand the feature spec to the developer. Weeks later, we get the first version of the app and try it out. For the first time, we are walking through the software step by step, and we see massive holes. We write down our revision notes, and send it off to the developer to build version 2.

At the end of this whole process, after doing this a few times, we have a pretty good solution. The trouble is that the development cycle is really long. Each loop takes weeks or months.

While discovering the inherent complexity of your app is a necessity,

Going back and forth with a developer is an expensive and time-consuming way to discover this complexity.

What we want to do is this:

We want to be able to discover as much of this complexity before we write a line of code. We want to peel back the layers to discover a more robust feature list, and we want to do this in a quick and inexpensive way.

Instead of iterating on the app, we want to iterate on the spec.

We want to go from this process:

To this process:

To do this effectively, we need two things. One, we need a way to represent the feature list, so that it can be iterated on. Two, we need a way to interrogate this feature list effectively.

Let’s talk about how to do this.

Mapping out the features

We want to stop thinking of this in business terms and start thinking of this like a user. That is the level at which we will have to deal with the complexity of the software.

First, make a list of user objectives. What does the user want to achieve using your app?

Next, you need to map out the user flow. Walk through every step that a user takes to achieve each of your objectives. Record those steps.

What we are building is a map of how users flow through the app.

There are many ways to do this. You could just make a simple outline of each step taken. You could draw up mockups of each screen the user goes through. You could draw a flowchart, either on paper or using software.

Whichever you pick, make sure to prioritize speed. Unless you’re an expert at Photoshop, put it away and pick up pen and paper.

Make sure every screen is represented in your outline. For example, if a particular step goes through multiple screens in the app, split it up. We want to tease out more details, not less.

This process alone should start surfacing some of the complexity of your app. Now, we drill deeper.

Question everything

Asking questions like we did above can be quite open-ended, like we did above. It can be hard to know where to begin.

To start you off, I’ve put together a list of common questions that apply to most software. These questions are arranged into four broad categories that you can use as a framework when attacking your outline.

Look at each step of your outline, and ask yourself every question on this list. If the answers reveal new steps, add them to your outline.

Here’s the list:

User inputs

These questions are about information that goes from the user to the product.

• What information does the user input?
• Is this input open ended or interactive?
• Open ended examples: free-form text input, choosing an image to upload, recording a video.
• Interactive examples: text input into a search form that displays results to pick from, choosing an address on a map, picking from a predefined collection of options.
• Is there any input that would be considered invalid, and how does the app deal with it?
• Are there any passive inputs? Most common example: user location via GPS.
• What information does the product need from a new user? Is there a way for the user to change this later?

Information presented to the user

If the previous section was inputs, you can think of this one as outputs.

• What information is shown to the user on this screen?
• How is this information shown? Examples: text, images, maps, lists, graphs.
• Does the information need to be sorted in some way? Examples: recency, distance from user, relevance.
• Does the product send any active outward communications? Examples: emails, push notifications, SMS messages.

Interactions between the pieces

These questions are about how different pieces of the experience (users, the product, other services) interact with one another.

• Do users interact with each other? Examples: messaging, adding as friend, “liking” or tagging of other users’ content.
• Do the users interact with external services? Examples: payment services, shipping tracking, social login.
• Does the product interact with external services? Examples: location lookup services, weather APIs, social networks (“Your friend X just joined! Check out their profile!”).

Business owner functionality

Questions about what you, the business owner, need from the product.

• Do you need to be sent alerts or summaries, via email or some other medium?
• Do you need a specialized system for yourself or employees? Examples: driver app for your delivery staff, real-time order management interface for your kitchen staff.
• Do you need an interface for manually approving users/content?
• Do you need a content moderation system? A way for your admins to easily remove user content, disable user accounts.

That’s… a lot of questions.

Yep! And it’s really just the common ones that people often forget early in the process. There are many more that will be specific to your product, and that you’ll uncover over time.

Bringing it all together

This process of building an outline and interrogating it will tell you what’s missing. Repeat this process a few times, and you should have a pretty solid feature list that you are much more confident about.

You know, I was really hoping you had this one weird trick to fix my project. This sounds like a lot of work.

It really is. But this is work you would have done at some point in the process anyway.

Remember why we are doing this. Software features have hidden complexity that you will need to uncover no matter what. You just get to choose when you peel the onion.

Essentially, you have two choices:

1. You can discover user objectives, build a mockup or flowchart, question your own assumptions, and peel back the layers like we did above. Or,
2. You can get a developer to start building your app and uncover these issues with each version, one week at a time. After several months, you will discover the exact same things you could have figured out in a week with pencil and paper.

This is why software projects blow up and try to eat you alive.

So, go and discover your features. Build your outline and question your assumptions. Uncover the bulk of your problems quickly and cheaply.

Save yourself the stress and pain, release your product out into the world, and let it bloom.

Do onions even bloom? Your metaphors are stupid.

Oh touché.

If you found this article useful, you can help me out by sharing it! You can also subscribe to my mailing list to be sure not to miss my next article.

This story is published in The Startup, Medium’s leading publication for entrepreneurs and startups.

Join +12,417 people who get the top stories here.

Your app is an onion: Why software projects spiral out of control was originally published in The Startup on Medium, where people are continuing the conversation by highlighting and responding to this story.

“So… what is it you do, exactly?”

She starts to tell you, and you kind of get it. You know what a user interface is. You’ve used apps. Facebook’s a website. You get it. You nod along to her answer, and the two of you part ways. Awkwardness avoided.

But sometimes, you meet one of the… other ones. The server engineers. The backend developers. Not knowing what you’re getting yourself into, you enquire about the arcane ways of their kind.

Listening to these foreign words, a barrage of questions runs through your mind. “What is an API? When do we use databases? And who is Jason?”

What your engineer friend is talking about is a server. Maybe you’ve heard the term before, but don’t really have any idea what that is. Today, we change that.

Down the rabbit hole

When we use an app, all we see is the app. Our experience as a consumer starts and ends there. It seem natural to think of the entire experience as “the app”.

Truth is, the app is just what’s known as the front end. It makes up the entirety of the user experience. But that is not all there is to the functioning of an app.

Say you send me a message on a messaging platform, such as Whatsapp. It appears like the message goes from your phone to mine. But let’s take a closer look at this. Let’s say that you send the message when my phone is switched off. You then switch your phone off, and I switch mine on. I still get the message, even though our phones were never running at the same time!

Clearly, we’re missing something here.

The missing component is the back end, or server.

Servers

A server is a computer that is connected to the internet and runs all the time.

A server has two main roles: storing data, and facilitating communication.

So, when you sent that Whatsapp message, the app on your phone sends the message up to the server, where it is stored. When my app checks in with the server, the server sends along any stored messages.

A server is where an app “phones home” to.

When the app needs any information, it asks the server. When the app needs to talk to another user on the app, the server is what facilitates that connection.

The terms server, backend, and API are often used interchangeably.

Storage

A primary responsibility of a server is storing things.

These include files, such as photos, videos, and documents. The server stores them in a structured way, similar to folders on your computer, so that they can then be accessed by apps.

A server also stores information. All apps have bits of information that are important to how the app works.

You can think of this as a bunch of spreadsheets. For example, apps need to store user information, and their login details, so that they can be authenticated. Your app might be a restaurant directory, in which case the server would store information about each restaurant.

Along with information, a server also records the relationships between bits of info. For example, if a user “likes” a restaurant on the app, the server remembers that relationship between that user and that restaurant.

This lets us get answers to many questions, such as:

“How many people have liked this restaurant?”

“What restaurants has this user liked?”

“What cuisines do both of these users like in common?”

Information and the relationships between them are stored in databases. There are many different kinds of databases, but they all have a few core features:

• They can store information
• They can store the relationship between bits of information
• They can be queried about the information, whereupon they respond with individual bits of information, or information in bulk, depending on what we asked for

There are too many different types of databases for me to list, and they each have their own advantages and disadvantages. If you hear someone use terms like “SQL” (pronounced ‘Sequel’), “MongoDB”, “CouchDB”, “Redis”, they’re talking about a database.

Communication

A key responsibility of a server is communicating with the app and with other servers.

Many activities on an app require communication with the server. For example, if the user searches for something, the search terms are sent up to the server, which responds with the results. If the user sends a message to another user, the message goes up to the server. It is then sent to the other user’s app, often in the form of a push notification.

The interface that the server provides, so that apps can talk to them, is often referred to as an API. The individual functions on the interface can be referred to as endpoints.

These communications can look bizarre to the uninitiated. The two most common communication formats are JSON and XML.

At first glance, these formats can seem really hard to read.

The thing to keep in mind is that a server is just a computer, like your laptop or your phone. An app on your phone takes in user inputs, in the form of touch or voice. It processes that information. It then gives outputs, in the form of images on the screen. A phone is a computer that talks to humans, and so the inputs and outputs are human friendly.

A server is a computer that only talks to other computers.

Humans express meaning using things like font size, text color, and layout. But these are not meaningful to a computer.

Server communications happen in formats that are easy for another computer to parse and understand.

Server application

Much in the same way you would write an app to run on your phone, you need an application running on the server. A server application is built using a server-side framework. Popular options include Ruby on Rails, PHP, ASP.NET, Java, Node.js.

The API is the gateway to your server, and apps know to call there. Your database stores all your information. Your server application is the “brain” that holds everything together.

It listens and responds to requests that come in on the API, it adds and retrieves information from the database, and it makes decisions. For example, when the app sends up login information, the request comes in through the API, and the correct login info is stored in the database. But it’s the server application’s job to compare the two, and respond appropriately to the app via the API.

Hardware

When someone says “server”, this might be what pops into your mind. Racks of blinking lights in a locked room, waiting for Tom Cruise to rappel down from the ceiling and steal from.

Many larger companies do own such servers. Facebook and Google have hundreds of servers all over the world. When you are running a huge service with millions of users, running your own servers can be cheaper and offer better performance.

Instead of running their own physical servers, many developers use some kind of cloud service. Services like Amazon Web Services, Azure and Digital Ocean can provide “virtual servers”. These services own and maintain the hardware, and the developer just uploads the server application.

There are even some Backend as a Service providers, letting you have a simple backend without writing a server application yourself.

Do all apps need a backend?

Most apps you see probably have a backend component to them. There are definitely apps that don’t, like certain productivity apps. An easy way to figure out if there’s some kind of backend:

If you switched to airplane mode, would the app still work?

If the answer is no, there’s probably a backend server involved somewhere.

Hopefully you now have a better grasp of what the backend of an app looks like. Maybe, the next time you meet your friend, you might even casually drop “JSON” into the conversation.

If you found this article useful, you can help me out by sharing it! You can also subscribe to my mailing list to be sure not to miss my next article.

The non-techie’s guide to servers was originally published in HackerNoon.com on Medium, where people are continuing the conversation by highlighting and responding to this story.

When I plan lessons, I start by outlining the main topics to be covered in a course. But it’s important to then switch away from content to skills.

The students have turned up at my course to learn iOS development. Ostensibly. But the reason they’ve really come here is to become iOS developers. And those are two different things. Knowing the difference means that I need to focus on building their skills as developers.

Skill #1: Knowing Things

This is the most obvious one. If you want to develop in a certain platform, there are things you need to know.

What’s a view controller? What’s a table view? What’s an if statement?

Because this is the most obvious skill, instructors are usually pretty good with this. We know we need to start with simpler concepts and work our way up that ladder. We know we need to test their understanding by asking questions.

Knowing the content is important, but it’s not the only skill!

Skill #2: Problem Decomposition

It’s one thing to know what an “if statement” is. Knowing that it needs to be used is a whole other skill.

When we teach and run through sample problems on the board, students are generally going to understand what we say. We can take the problem, break it down into steps, and explain each line of code.

The same student is often going to be sitting at their desk, stunned, staring at a blank file on their computer.

Understanding the steps on the board doesn’t mean they can look at a problem and identify the sub problems.

People learning a foreign language often hit the same barrier. A person who can read reasonably well can still have enormous difficulty understanding a native speaker speaking. Why? They know the words, don’t they?

Here’s why. If a native speaker says “An apple a day”, what you hear is “An apple a day”. Here’s what a learner hears:

“A na pe le de”

Those are the syllables. And none of those are words. Fluent speakers instantly break that into:

“ A n|a pe l|e | de ”
“  An  apple a   day”

But it’s hard for learners to see the split. Even if they know all the words, they can’t see the words in the collection of syllables.

Programming students are the same way. They know all the ingredients that make up the solution. But it’s hard for them to see which pieces need to be used in this particular problem. It’s our job to teach them to see those seams.

Skill #3: Muscle Memory

Every programming instructor has seen students look at their screens filled with red, having missed a semicolon or comma somewhere. This is normal, of course. Eventually students will start getting the details right.

The way to speed this up is to build up what I like to think of as their muscle memory. This is distinct from knowing the content, or breaking down a problem. This is about actually implementing the solution. The writing of the code, or the creation of the layout.

It’s not enough for your brain to know how to code; your fingers need to know how to code as well.

This might sound a bit strange. We tell our fingers what to do. Our fingers don’t move independently. But don’t they seem to, sometimes? You automatically close a bracket that you just opened. You automatically fix the indentation on a new line. There are several tasks that are almost instinctive.

It’s like speaking a language. We don’t craft every word in our brain and then instruct our tongues. We think thoughts, and the words find their way out into the world. Of course, we can be more deliberate in our words and enunciation if we need to be. But for 90% of communication, we focus on the thoughts, not the production of sound.

It’s the difference between planning and execution. Both are important, and they are distinct skills.

When planning lessons, keep an eye out for which of these skills you’re training. All three are important.

Because here’s the thing. Once you deliver on skill 1, students are going to be satisfied. They showed up to learn X, and you taught them X. The content has been delivered.

But they’re still not going to be effective developers. And they’re going to blame themselves. When they’re stuck with a problem and you then show them the solution, they’re going to feel stupid.

“Of course I should have known how to do this. He used variables and ifs. I already know this stuff. I must not be cut out for programming.”

“How does he always know which bracket is missing in my code, and I can never see it? I guess I don’t have an eye for this sort of thing.”

It’s your duty as an educator to give students what they need, not just what it says on the title of the course.

If we keep an eye on all three skills, we’ll be doing just that.

Connect Deeper

If you resonated with this article, do subscribe to my personal blog! You will be notified of any future articles I write on tech education.