Every creative person has a project — or at least a project idea.

FindCollabs is a place to share your projects, whether those projects are new or old. Your project doesn’t even have to have code associated with it yet — it can just be an idea, like a weather prediction system.

Your project can be anything creative — a game, or an animation, or a song.

One project I’m working on is a FindCollabs Theme Song. Two collaborators have worked on the project, Adam and AJ. I already gave Adam a review (he did a great job). AJ’s review is pending.

The workflow for submitting a project is easy. As an example, I’ll go through a project that I want to share with the FindCollabs community.

This is Escrow, a social betting game I invented. It’s not the prettiest game in the world, but it has some cool ideas. Maybe other people will want to contribute to it.

Let’s add it to FindCollabs!

I added the link to my GitHub repository and a live version of the app so people can test it out. Now I’ll create some roles, so people know how they could contribute to the game.

That’s it! Now my project is created on FindCollabs. People who come to FindCollabs will see it in the feed.

When you click into a project, you are taken to an interface that allows you to chat and interact with the project. I’m already getting some feedback from Vynce.

FindCollabs is a place for you to share your projects and find collaborators.

The projects can be anything creative. When people collaborate online, cool stuff gets built.

One project on FindCollabs is SoftwareDaily.com, a site that is built to be a better, more flexible skin for SoftwareEngineeringDaily.com. We have a popular podcast, but the content is difficult to index, search, and discover. We have outgrown WordPress and podcast players.

The community has been building the Software Daily platform for a few years, and FindCollabs gives us a platform to work within.

On FindCollabs, you can post new projects and old projects. You can post a rough idea or a detailed product spec.

When you share your projects, other people can get a sense for who you are and how you like to work. Here’s my profile.

FindCollabs is having an online hackathon with $5000 in prizes. If you have some cool projects you are working on, put them on FindCollabs and be entered to win.

The judges are investors, artists, and engineers. They understand the creative process. Entrees to the hackathon will be judged based on vision, execution, creativity, and teamwork.

If you don’t have any projects to post, you can join an existing project.

We look forward to seeing you on FindCollabs!

FindCollabs: Share Your Open Source Projects and Find Collaborators was originally published in HackerNoon.com on Medium, where people are continuing the conversation by highlighting and responding to this story.

Writing music with on-demand musicians

The gig economy is a revolutionary tool for artists.

Creative services platforms like Fiverr bring together the demand and supply for artistic labor.

Artists with a big vision need to hire collaborators. Artists who want to practice their artistic craft while earning some spare cash need gigs to work on. Fiverr brings the two sides together, acting as an escrow and mediating the occasional dispute.

New artistic tools give rise to new art forms. Photoshop created new types of visual art and memes. Unity made 3-D model design accessible, leading to new games. Ableton simplified the ability to make electronic dance music, birthing new genres.

The gig economy is a powerful new artistic tool because it unlocks paid collaboration.

In this post, we’ll explore why the gig economy is so useful for artists, and how to use it to bring a big artistic vision into reality.

Creative Gigs

The gig economy is widely used for physical tasks such as driving, grocery delivery, and house cleaning.

There is also a growing adoption of basic digital services such as transcription, data entry, and web scraping. Even white collar knowledge jobs like medicine, law, and accounting are moving onto the gig economy.

Platforms like Uber, Instacart, and UpCounsel drive down costs and improve market efficiency by aggregating supply and demand on a digital platform. You can use the gig economy for veterinary house calls, flower delivery, and podcast editing.

For transactional, well-defined tasks, the gig economy is clearly a game-changer. But what about creative work?

Writers, designers, animators, and musicians can be hired through the gig economy. Thousands of artists across the world make a living online while practicing what they love. They are cheap to hire, and many of them are quite good at what they do.

How can art change with this dense aggregation of supply and demand for artistic work?

I decided to experiment with how the gig economy might be used for music production. Over the last year, I hired ten different vocalists from Fiverr to perform across an album that I wrote. I also hired three sound engineers to mix and master the tracks.

Writing this album was extremely fun and satisfying. Being able to pay talented strangers to work with me over the Internet felt futuristic.

My perspective on art has been changed by Fiverr to the degree that my perspective on car ownership was changed by Uber. The gig economy dramatically lowers the cost of ambitious artistic projects.

Paid Collaboration

Whoever you are, the Internet invites you to create ambitious art. On the Internet there are free resources for learning to make art, free tools for creating your art, and free social networks for distributing your art.

The gig economy fills a missing piece: paid collaboration.

Whether you are creating a television show, a novel, or a business, you can use the gig economy as a vast talent pool to help you solve problems.

Why is paid collaboration so important? Because artists normally want to work on their own art. Art is a solitary pursuit. We make art when we are by ourselves. We use art to channel our individual emotions into a book, song, or painting.

The insular nature of art makes it hard to find artistic collaborators. Everyone who could collaborate with you has their own art that they are working on. Why would they want to help you with your project?

The best works of art involve multiple people. A writer needs an editor. A singer needs a songwriter. A software engineer needs a UI designer.

If you run a business, you can afford to hire a full-time employee to help you with paid collaboration. But if you are creating an art project on the side, you don’t have a big budget to allocate.

It’s hard to make good art by yourself. The gig economy lets you find affordable collaborators to get artistic leverage.

A Pop Workflow

I have written music on the computer for fifteen years. Since Distrokid made it possible to release music to Spotify, I have published five albums. None of it was great music, but I’ve been steadily improving.

I am like many digital artists. I am inspired by the power of the Internet to create and distribute art. I want to share my sound with the world, and I want the quality of my art to be good enough for people to connect to it.

From 2003 to 2017 I mostly worked in isolation. I didn’t seek out mentors, or solicit constructive feedback, or look for collaborators. And I didn’t try to learn music theory.

For this new album, I wanted to level up. I wanted to use the Internet to educate myself on how to excel as a musician. And I wanted to work with singers who could complement my productions.

My goal for the album was to create catchy modern pop music.

My musical role model is Max Martin, the music producer who is able to work productively with almost anyone. He has created many of the hits you have heard from the last two decades, from Britney Spears to Bon Jovi to Taylor Swift.

Max knows his strengths and weaknesses. His humility and leadership allow him to assemble an elite musical team. It’s no accident he has continued to create pop masterpieces longer than almost anyone in history.

I envisioned each gig economy vocalist as an A-list pop star whom I was lucky to borrow some time with. It was my job to bring out the best performance in these singers, by providing them incredible instrumentals and memorable lyrics.

With this in mind, I needed to have fully composed songs, ready to perform for each singer. I wanted the singers to know that I respected their time and their talent. In return, they would provide top-notch performances.

To compose these songs, I used a digital audio workstation (DAW). A DAW is like Photoshop for music. It allows the producer to compose melodies, create instruments, record vocalists, mix the tracks, and do everything else involved in creating music.

Many mainstream hits are made by “bedroom musicians” working with only a DAW and a set of headphones.

The DAW is a rich palette of tools for writing music. It is empowering, but can also be overwhelming. It is easy to get lost in the details, spending hours at a time tweaking a minor detail of a song.

When I started this album, my songwriting was not efficient. My music wasn’t catchy, and it lacked the clear structure and addictiveness of pop music. I didn’t know how to write lyrics that would stick in your head.

With the help of Google, I took a crash course in composing modern pop music.

Gary Ewer’s Secrets of Songwriting is invaluable. Gary covers the music theory and the algorithmic structure of pop songwriting in a clear, approachable way.

There is a reason that pop songs are popular. The pop format is scientific. Great pop music comforts us with its familiarity, but provokes us with slight deviations from what we have heard before.

“You have a really basic arrangement, a very simple vocal, very simple transitions. It’s almost like a formula where you replace the lyrics and the music, and keep the drums the same.” — Calvin Harris

There are many different formulas for architecting a pop song. It is important to understand the basic concepts of chorus and verse. These represent two distinct areas that almost every pop song has. Songs with more variety might have a pre-chorus and/or bridge section in addition to chorus and verse.

Music theory allows the songwriter to build these distinct sections in a structure that makes sense to the listener. From Gary’s articles, I finally grasped the importance of the “circle of fifths”, which is as useful to songwriting as the periodic table is to chemistry.

Now I keep a window with the circle of fifths open at all times when I write music.

Gary emphasizes the use of “hooks”. A hook is a musical structure that grabs the listener and sticks with them. A hook can appear in the chorus or verse. Some songs have one incredible hook, others have several different hooks.

As I learned about music theory, I looked for videos that could illustrate the pop theory which Gary talks about in his articles. I found a video series from Pyramind called “Breakdowns”, in which Matt Donner breaks down the theory of modern pop/electronic music.

I found Matt’s advice to be so useful that I booked private lessons with him, which I recommend to anyone who is looking for an electronic music teacher.

Matt’s videos emphasize how to be productive as an electronic musician. If you want to sit down and get a song fully written in four hours, how do you divide your time? The DAW provides you with a paralyzing array of choices. What are the priorities?

My priority became the hook. A pop song lives and dies by the hook. A song without a hook cannot succeed as a pop song. Conversely, a good hook can compensate for an otherwise boring song.

For me, finding the hook was the hardest part of writing a pop song. During the months writing this album, there were many days in which I did not find a hook. But with patience and persistence, a catchy melody always seemed to find its way onto the screen eventually.

Once I found a melodic hook, I would start writing lyrics around it. If it was a chorus hook, I would write lyrics exclusively for the four or eight bars of chorus. The same was true if I had found a verse hook.

Alex Forbes insists that writing lyrics is about getting in touch with your subconscious and reflecting your true state of mind onto the page. Otherwise the lyrics will sound hollow and the song will lack the vigor it needs to succeed as a pop song.

Pop lyrics can take many forms — but they must be catchy.

Pop lyrics can use both vague, abstract phrases and evocative, specific imagery. Lyrics can use rhyme, slant rhyme, and extended metaphors. Any literary technique that can work in a book can work in a song.

But the lyrics must be catchy.

The words must be fun to say. Certain phrases must make your spine tingle. Discovering catchy lyrics is as important as finding a hook.

With a hook and a section of lyrics around that hook, the rest of the song fell into place naturally. The articles from Gary Ewer provided a great system for building a fully formed song architecture around a catchy hook.

This is where the circle of fifths again came into play. The circle of fifths provides a framework for moving between keys and developing a chord progression that has a different sound in the chorus and verse.

The circle of fifths can be used to map out the specific chords within a section of a song. It can also be used to define how to move from one section of a song to another. The sequence of sections within a song is called the “song form” or “song structure”.

The different sections within a song’s form are often enumerated with letters such as A, B, and C (not to be confused with the keys of A, B, and C). Each letter denotes a melodic section of a song.

The “AAA…” or “Strophic” song structure is the simplest. In “AAA…” the chords in the chorus are repeated in the verse.

“AAA…” can be a comfort zone. In this album I needed to swallow the bitter pill of learning the basics of the circle of fifths so that I could explore more complex structures like “ABABC…”.

HookTheory is another invaluable tool. HookTheory allows you to see and hear the chord progressions behind your favorite songs. Using HookTheory alongside the circle of fifths is a great way to see how music theory underpins the best pop songs.

As I wrote my new album, mornings were spent reading tutorials from Gary Ewer. In the afternoon I watched videos from Pyramind. I ate dinner in front of my computer, playing with HookTheory and seeing how Zedd’s chord progressions compare to those of The Eagles.

I love the Internet for this reason: you can totally engross yourself in a topic. Multiple forms of educational material can be tied together to customize your own ideal learning environment.

Working With Singers

Each song started with an instrumental hook. After the hook, I wrote the rest of the instrumentals concurrently with writing the lyrics. When I finished writing all the instrumentals and all the lyrics for a song, I recorded myself singing the lyrics.

Once I had lyrics recorded with the song, I had a first draft of the song finished. I exported the song to mp3 and put it in a Dropbox folder that was synced with my phone.

Creating a pop song usually takes at least a few days. Sometimes it takes as long as a month. When Max Martin is working on a song, he regularly bounces the song project to mp3. He listens to the song on earbuds, laptop speakers, and car stereo systems.

We listen to pop all over the place, so it is important to hear a song in multiple contexts before finalizing it.

I like to listen to music on cheap bluetooth headphones while running outside. It’s important that my own music sounds good in that context. I focused intently on the quality of my work-in-progress music as I jogged through the hills of Pacific Heights.

Listening to myself sing the lyrics, I thought about what type of singer would be best for that particular song. Should it be an angelic female voice? A baritone male? A youthful boy band-type?

To find vocalists, I used Fiverr. There are other platforms such as UpWork, SoundBetter, and Vocalizr. These platforms all have their advantages, but Fiverr seems to have the best selection of singers at low cost, and the software and UI is top notch.

Singers on Fiverr have a profile page that shows them demoing their skills. There is a high correlation between the quality of a singer’s demo and the quality of the end result they will deliver.

When I decided on a singer, I messaged them an offer including the written lyrics and the version of the song sung by me.

When a singer accepts an offer on Fiverr, the singer commits to a deadline. The customer (me in this case) gets charged for the cost of the gig, and that charge sits in escrow until the performance is delivered and the transaction is finalized.

As I wrote this album, I tried to focus on one song at a time if possible. I wanted to complete a song in its entirety before switching to a new song.

Unfortunately, the best singers usually have a backlog of orders. From the time of my initial message, it would take 4–7 days for the singer to get to my lyrics and perform them. This created a bottleneck in the workflow for an individual song.

In practice, I was usually working on 2–3 songs at any given time. To manage the album, I used a Trello board to visualize the progression of songs through different phases of my workflow.

When a singer finished their lyrics, they uploaded the audio file to Fiverr. I downloaded it, loaded it into my DAW, and listened to their performance. Usually there would be 2–3 areas of their delivery where I asked for revisions.

Fiverr transactions have this “revision” feature built in. I never felt uncomfortable asking a singer to do another take, or to give another vocal track so that I could overdub them.

When I did make a revision request, I learned to be extremely specific about what I was asking for. The last thing a Fiverr singer wants to hear is a phrase like “use your best judgment” or “take creative liberty”.

Telling a singer to “take creative liberty” adds subjectivity to the transaction. Subjectivity creates the chance of a dispute over quality.

A customer could say that the singer didn’t do a good job, and demand a refund — and in the gig economy, the customer is always right. The singers want rigid specifications for how to sing a song to protect themselves from doing wasted work.

Cost

A vocal performance on a typical three minute pop song usually cost me $70 — $140.

You can also hire very cheap singers, but you usually get what you pay for. I paid $10 for a Nigerian R&B singer with an amazing voice, but unfortunately his cheap microphone degraded the audio quality too much.

A pop song demands a great vocalist who has put in the necessary practice. But the singer also has to be a good fit. On a few occasions, I hired an experienced vocalist whose performance did not end up fitting well with the song. There was some sunk cost of ~$400 across the album.

In some cases, I found vocalists who were radically underpriced. Two cases were the smooth female vocalist called leahhart and a superb Bahaman rapper called grafezzy. Each of these artists were about $40 per three minute engagement.

Finishing Touches

Eight months into my album, I had the vocals and instrumentals completed for sixteen songs, amounting to just under an hour of music. These tracks were not mixed.

Every pop song must have a good mix. The mix refers to the overall feeling of a track, including volume of the instruments, the panning, and effects such as reverb and compression.

When I wrote these songs, I was not focusing on the mix. This was a deliberate decision — I love working inside the mixer, toying around with effects and tweaking volume levels. But I only have so much time. Writing music in a DAW is such an expansive decision space that you must decide what you will not do.

I knew that I could save mixing until the end and find someone to help me with that part. The time and energy I saved avoiding the mix was allocated to songwriting. However, once the songs were all written and the vocals were recorded, it was time to spend some time on the mix.

I hired two studio engineers from SoundBetter ($400 each) and one from Fiverr ($50 per song) to mix and master three of my songs. All three mixes turned out well — but I decided that I wanted to be in the room for the mixing/mastering process on the rest of the tracks. There are too many small, creative decisions that I wanted to pass judgment on.

Studio time with an engineer at Pyramind was cheap, and it would allow me to hear the music on nice speakers in a soundproofed room. I spent a few days in their Folsom Street studio called “The Vault” with an engineer named Sam.

Once we finished the mix, I did a final test, running outside listening to the finished album on my bluetooth headset, to make sure it sounded good away from a controlled studio environment.

After ten months the album was finished and ready to be uploaded to Distrokid. The complete album is available on Spotify, YouTube, and other music platforms.

The Sell Side

From the buyer‘s perspective, the creative gig economy is a huge win. The effects on the seller side are positive too.

One of the singers I worked with said this: “My personal thought on the ‘gig economy’ is it a great resource to make a living while also doing what you enjoy. I would way rather be singing all kinds of random music jobs (something I like and am pretty good at) than be bussing tables or something equally as not interesting to me.”

“My personal thought on the ‘gig economy’ is it a great resource to make a living while also doing what you enjoy.” -Fiverr Singer

“It also helps me improve my efficiency in recording, in turn making it faster when I go into the studio to record my own music. In a sense it levels the playing field with pricing (a pro for some, con for others), helps buyers choose creative people with seemingly quality work and ethic (via ratings) and opens up connections that would never be made otherwise.”

The gig economy is a place for people to build skills while working for money. In this way, the gig economy also functions like a trade school with no tuition. Teenagers on Fiverr have helped me with creative animation and podcast editing. They are receptive to feedback and hungry to learn.

For workers who already have in-demand skills, the gig economy can offer superior working conditions. Many elite tech workers subject themselves to an unhappy 2-hour daily commute, when platforms like Toptal offer work from home at a high salary.

The gig economy is a dense aggregation of buyers and sellers that allows for rapid price discovery and an efficient market.

In some markets, we have seen the negative consequences of the gig economy’s efficiency. On-demand drivers perform 12-hour shifts to meet a quota. Food delivery workers ride bikes through snow to deliver pad thai while it’s still hot.

The most grueling gig economy work is within the commoditized fields. If you are driving an Uber or delivering groceries for Instacart, it’s very hard to differentiate yourself. You cannot escape the ruthless downward pricing pressure of fungible labor.

On the other hand, there is no salary ceiling in fields like graphic design and singing. The artistic work is more fun to do, and there is always upward mobility. Artists have very real pricing power.

Collaboration Leads To Better Art

In 2003, The Postal Service came out with Give Up, an electronic pop album that has stood the test of time.

The band’s name was a reference to the fact that the album was collaborated on over physical mail, with the artists sending digital audio tapes to each other.

As recently as 2003, the most efficient way to collaborate on music was to send tapes by mail. Fifteen years later, online artistic collaboration still feels immature.

The most highly developed domain for artistic collaboration is software engineering. On GitHub, thousands of users make shared contributions to software. Some of the world’s most impactful software is the result of these huge crowds of developers building software for free, in an environment of managed chaos.

In software engineering, decentralized collaboration has led to revolutionary technologies such as Linux and Bitcoin. Strangers work together to build technology that would be impossible for any one person to develop.

Linux and Bitcoin were started by individuals, but the online collaboration tools for software made it possible for the number of contributors to scale.

The art world has not yet been so affected by online collaboration. The songs we listen to are produced by a few people working in a studio. Movies are produced by monolithic studios.

This is an opening for a new generation of artists to rethink how collaborative art can be made with the Internet.

Newer tools like Slack, Asana, Dropbox, and Splice have changed the way artists can manage their projects. The gig economy gives us access to paid collaborators.

Over time, these tools will reshape the art that we make and consume. We will see more bedroom musicians and film directors who work out of coffee shops. Decentralized artistic teams will use software to bring their projects to life.

There has never been a better time to be an artist with a big, ambitious vision.

Singing In The Gig Economy was originally published in HackerNoon.com on Medium, where people are continuing the conversation by highlighting and responding to this story.

To listen to the podcast version of this episode, view it on Software Daily.

Most new internet businesses started in the foreseeable future will leverage Kubernetes (whether they realize it or not). Many old applications are migrating to Kubernetes too.

Before Kubernetes, there was no standardization around a specific distributed systems platform. Just like Linux became the standard server-side operating system for a single node, Kubernetes has become the standard way to orchestrate all of the nodes in your application.

With Kubernetes, distributed systems tools can have network effects. Every time someone builds a new tool for Kubernetes, it makes all the other tools better. And it further cements Kubernetes as the standard.

Google, Microsoft, Amazon, and IBM each have a Kubernetes-as-a-service offering, making it easier to shift infrastructure between the major cloud providers. We are likely to see Digital Ocean, Heroku, and longer tail cloud providers offer a managed, hosted Kubernetes eventually.

In this editorial, I explore the following questions:

• Why is this happening?
• What are the implications for developers?
• How are cloud providers affected?
• What are the new business models that are possible in a Kubernetes-standardized world?

Software Standards

Standardized software platforms are both good and bad.

Standards allow developers to have expectations around how their software will run. If a developer builds something for a standardized platform, they can make smart estimations about the total addressable market for that piece of software.

If you write a program in JavaScript, you know that it will run in everyone’s browser. If you create a game for iOS, you know that everyone with an iPhone will be able to download it. If you build a tool for profiling garbage collection in .NET, you know that there is a large community of Windows developers with memory issues who can buy your software.

Standardized proprietary platforms can lead to massive profit returns for the platform provider. In 1995, Windows was such a good platform that Microsoft could sell a CD in a cardboard box for $100. In 2018, the iPhone is so good that Apple can take 30% from all app sales on its platform.

Proprietary standards lead to fragmentation.

Your iPhone app does not run on my Kindle Fire. I can’t use your Snapchat augmented reality sticker on Facebook Messenger. My favorite digital audio workstation only runs on Windows — so I have to keep a Windows computer around just to make music.

When developers see this fragmentation, they groan. They imagine the greedy capitalists, who are making money at the expense of software quality. Developers think, “why can’t we all just get along? Why can’t we have things be open and free?”

Developers think, “we don’t need proprietary standards. We can have open standards.”

Growth of Apache, part of the LAMP (Linux, Apache, MySQL, PHP) Stack

This happened with Linux. These days, new server side applications are mostly in Linux. There was a time when that was controversial (see the far left hand side of chart above).

More recently, we saw a newer open standard with Docker. Docker gave us an open, standardized way of packaging, deploying, and distributing a single node. This was hugely valuable! But for all of the big problems that Docker solved, it highlighted a new problem — how should we be orchestrating these nodes together?

After all — your application is not just a single node. You know you want to be deploying a Docker container — but how are your containers communicating with each other? How are you scaling up instances of these containers? How are you routing traffic between container instances?

Container Orchestration

After Docker became popular, a scramble of open source projects and proprietary platforms emerged to solve the problem of container orchestration.

Mesos, Docker Swarm, and Kubernetes each offered a different set of abstractions for managing containers. Amazon ECS offered a proprietary managed service that took care of installation and scaling of Docker containers for AWS users.

Some developers did not adopt any orchestration platform, and used BASH, Puppet, Chef, and other tools to script their deployments.

Whether a developer was deploying their container by using an orchestration platform or a script, Docker sped up development, and made things more harmonious between developers and operations.

As more developers deployed containers with Docker, the importance of the orchestration platform was becoming clear. A container is as fundamental to a distributed system as an object is to an object oriented program. Everyone wanted to be using a container platform, but there was a struggle between these platforms, and it was hard to see which would come out on top — or if it would be a decades long struggle, like iOS vs. Android.

This struggle between the different container orchestration platforms was causing fragmentation — not because any of the popular orchestration frameworks were proprietary (Swarm, Kubernetes, and Mesos are all open source), but because each container orchestration community had invested so much in their own system.

So, from 2013–2016, there was some anxiety among Docker power users. Choosing a container orchestration framework is a huge bet — and if you chose the wrong orchestration system, it would be like opening a movie store and choosing HD DVD over Blu-ray.

The war between container orchestrators felt like a winner-take-all affair.

And as in any war, there was a fog that was hard to see through. When I was reporting on container orchestration wars, I recorded podcast after podcast with container orchestration experts, where I would ask some form of the question, “so, which container orchestration system is going to win?”

I did this until someone I respect told me that asking about who was going to “win the orchestration wars” was a less interesting question than evaluating the technical tradeoffs between the orchestrators.

Looking back, I regret buying into the narrative of a war between container orchestrators.

As the debates about container orchestrators raged on, the smartest people in the room were mostly having calm, scientific discussions — even when reporters like me were getting worked up, and thinking that this was a story about tribalism.

The conflicts between container orchestrators were not about tribalism — they were more about differences of opinion, and developer ergonomics.

OK, maybe the container orchestration war wasn’t only about differences of opinion. There is a boatload of money to be made in this space. We are talking about contracts with billion dollar legacy organizations — banks, and telcos, and insurance companies — who are making their way onto the cloud.

If you are in the business of helping telcos move onto the winning platform, business is good. If you champion the wrong platform, you end up with a warehouse full of HD-DVDs.

The time where the conflict was the worst was near the end of 2016, when there were rumors about Docker potentially forking, so that Docker the company could change the Docker standard to fit better with their container orchestration system Docker Swarm — but even in those times, it would have made sense to be an optimist.

Creative destruction is painful, but it is a sign of progress — and in the struggle for container orchestration dominance, there was a ton of creative destruction.

And when the dust cleared around the end of 2016, Kubernetes was the clear winner.

Today, with Kubernetes becoming the safe choice, CIOs feel more comfortable adopting container orchestration at their enterprises — which makes vendors feel more comfortable investing in Kubernetes-specific tools to sell to those CIOs.

This brings us to the present:

• The open source developers are rowing in the same direction, excited about what to build.
• Major enterprises (both new and legacy) are buying into Kubernetes.
• The major cloud providers are ready with low-cost Kubernetes-as-a-service.
• The numerous vendors of monitoring, logging, security, and compliance software are thrilled because the underlying software stack that they have to integrate with is becoming more predictable.

Going Multicloud

Today, the most lucrative provider of proprietary backend developer infrastructure is Amazon Web Services. Developers do not view AWS resentfully, because AWS is innovative, empowering, and cheap. If you are paying AWS a lot of money, that probably means your business is doing well.

With AWS, developers do not feel the level of lock-in that was administered by the dominant proprietary platforms of the 90s. But there is some lock-in. Once you are deeply embedded in the AWS ecosystem, using services like DynamoDB, Amazon Elastic Container Service, or Amazon Kinesis, it becomes a daunting task to move away from Amazon.

With Kubernetes creating an open, common layer for infrastructure, it becomes theoretically possible to “lift and shift” your Kubernetes cluster from one cloud provider to another.

If you decided to lift-and-shift your application, you would have to rewrite parts of your application to stop using the Amazon-specific services (like Amazon S3). For example, if you wanted an S3 replacement that would run on any cloud, you could configure a Kubernetes cluster with Rook, and start to store objects on Rook with the same APIs that you would use to store them on S3.

This is a nice option to have, but I have not yet heard of anyone actually lifting and shifting their application away from a cloud — except for Dropbox, and their migration was so epic it took two and a half years.

Certainly there is someone out there other than Dropbox who spends so much money on Amazon S3 that they will consider spinning up their own object store, but it would be a huge effort to do such a migration.

Kubernetes probably won’t be a tool for widespread lifting and shifting any time soon. A more likely scenario is that Kubernetes will become a ubiquitous control plane, that enterprises will use on multiple clouds.

NodeJS is a useful analogy. Why do people like NodeJS for their server side applications? It’s not necessarily because Node is the fastest web server. It’s because people like to use the same language on both the client and the server. Just like NodeJS lets you move between your client and server code without having to switch languages, Kubernetes will let you switch between clouds without having to change how you think about operations.

On each cloud, you will have some custom application code running on Kubernetes that interacts with the managed services provided by that cloud.

Companies want to be multi-cloud — partly for disaster recovery, but also because there is actual upside to having access to managed services on the different clouds.

One emerging pattern is to split infrastructure between AWS for user traffic and Google Cloud for data engineering. One company that uses this pattern is Thumbtack:

At Thumbtack, the production infrastructure on AWS serves user requests. The log of transactions that occur get pushed from AWS to Google Cloud, where the data engineering occurs. On Google Cloud, the transaction records are queued in Cloud PubSub, a message queueing service. Those transactions are pulled off the queue and stored in BigQuery, a system for storage and querying of high volumes of data.

BigQuery is used as the data lake to pull from when orchestrating machine learning jobs. These machine learning jobs are run in Cloud Dataproc, a managed service that runs Apache Spark. After training a model in Google Cloud, that model is deployed on the AWS side, where it serves user traffic. On the Google Cloud side, the orchestration of these different managed services is done by Apache Airflow, an open source tool that is one of the few pieces of infrastructure that Thumbtack does have to manage themselves on Google Cloud.

Today, Thumbtack uses AWS for serving user requests and Google Cloud for data engineering and queueing in PubSub. Thumbtack trains its machine learning models in Google, and deploys them to AWS.

This is just the way things are today. Thumbtack might eventually use Google Cloud for user-facing services as well.

More companies will gradually move towards multiple clouds — and some of them will manage a unique Kubernetes cluster on each cloud.

You might have a GKE Kubernetes cluster on Google to orchestrate workloads between BigQuery, Cloud PubSub, and Google Cloud ML, and you might have an Amazon EKS cluster to orchestrate workloads between DynamoDB, Amazon Aurora, and your production NodeJS application.

Cloud providers are not replaceable commodities. The services provided by the different clouds will get increasingly exotic and differentiated. Enterprises will benefit from having access to the different services on the different cloud providers.

Distributed Systems Distribution

Services like Google BigQuery and AWS Redshift are popular because they give you a powerful, scalable, multi-node tool with a simple API. Developers often choose these managed services because they are so easy to use.

You don’t see these types of paid tools for single nodes. I don’t pay for NodeJS, or React, or Ruby on Rails.

Tools for a single node are much easier to operate than tools for a distributed system. It’s harder to deploy Hadoop across servers than to run a Ruby on Rails application on my laptop. With Kubernetes, this is going to change.

If you are using Kubernetes, you can use a distributed systems package manager called Helm. This is like npm for Kubernetes applications. If you are using Kubernetes, you can use Helm to easily install a complicated multi-node application, no matter what cloud provider you are on.

Here’s a description of Helm:

Helm helps you manage Kubernetes applications — Helm Charts helps you define, install, and upgrade even the most complex Kubernetes application. Charts are easy to create, version, share, and publish — so start using Helm and stop the copy-and-paste madness.

A package manager for distributed systems. Amazing! Let’s take a look at what I can install.

Distributed systems are hard to set up. Ask anyone who has set up their own Kafka cluster. Helm is going to make installing Kafka as easy as installing a new version of Photoshop on your MacBook. And you will be able to do this on any cloud.

Before Helm, the closest thing to a distributed systems package manager (that I am aware of) was the marketplace that you find on AWS or Azure, or the Google Cloud Launcher. Here again we see how proprietary software can lead to fragmentation. Before Helm, there was no standard, platform-agnostic way of one-click installing Kafka.

You can find a way to one-click install Kafka on AWS, Google, or Azure. But each of these installations had to be written independently, for that specific cloud provider. And to install Kafka on Digital Ocean, you need to follow a 10-step tutorial.

Helm represents a cross-platform system for distributing multi-node software on any Kubernetes instance. You could use an application installed with Helm in any cloud provider or on your own hardware. You could easily install Apache Spark or Cassandra — systems that are notoriously difficult to set up and operate.

Helm is a package manager for Kubernetes — but it also looks like the beginnings of an app store for Kubernetes. With an app store, you could sell software for Kubernetes.

What kind of software could you sell?

You could sell enterprise distributions of distributed systems platforms like Cloudera Hadoop, Databricks Spark, and Confluent Kafka. New monitoring systems like Prometheus and multi-node databases like Cassandra that are hard to install.

Maybe you could even sell higher-level, consumer-grade software like Zendesk.

The idea of a self-hosted Zendesk sounds crazy, but someone could build that, and sell it in the form of a proprietary binary, at the cost of a flat fee instead of a subscription. If I sell you a $99 Zendesk-for-Kubernetes, and you can easily run it on your Kubernetes cluster on AWS, you are going to end up saving a lot of money on support ticketing software.

Enterprises often run their own WordPress to manage a company blog. Is the software for Zendesk that much more complicated than WordPress? I don’t think so — but enterprises are much more scared of managing their own help desk software than managing their own blogging software.

I run a pretty small business but I subscribe to a LOT of different software-as-a-service tools. An expensive WordPress host, an expensive CRM for advertising sales, MailChimp for the newsletter. I pay for these services because they are super reliable and secure, and they are complex multi-node applications. I wouldn’t want to host my own. I wouldn’t want to support them. I don’t want to be responsible for technical troubleshooting when my newsletter fails to send. I want to run less software.

In contrast, I’m not afraid that my single-node software is going to break.

Software that I use from a single node tends to be much less expensive because I don’t have to buy it as a “service”. The software that I use to write music has a 1-time fixed cost. Photoshop has a 1-time fixed cost. I pay for the electricity to run my computer, but other than that I have no ongoing capital expense to run Photoshop.

When multi-node applications are as reliable as single-node applications, we will see changes in the pricing models.

Maybe someday I will be able to purchase a Zendesk-for-Kubernetes. The Zendesk-for-Kubernetes will give me everything I need from a help desk — it will spin up an email server, it will give me a web frontend to manage tickets. And if anything goes wrong, I can pay for support when I need it.

Metaparticle

With Kubernetes, it becomes easier to deploy and manage distributed applications. With Helm, it becomes easier to distribute those applications to other users. But it’s still pretty hard to develop distributed systems.

This was the focus of a recent CloudNative Con/KubeCon Keynote by Brendan Burns, called “This Job is Too Hard: Building New Tools, Patterns, and Paradigms to Democratize Distributed Systems Development”.

In his presentation, Brendan presented a project called Metaparticle. Metaparticle is a standard library for cloud-native development, with the goal of democratizing distributed systems. In an introduction to Metaparticle, Brendan wrote:

Cloud native development is bespoke, complicated and limited to a small number of expert developers. Metaparticle changes this by introducing a set of utilities in familiar programming languages that meet the developer where they are and enables them to begin to develop cloud-native systems using familiar language features.

Metaparticle builds on top of Kubernetes primitives to make distributed synchronization easier as well. It supplies language independent modules for locking and leader election as easy-to-use abstractions in familiar programming languages.

After decades of distributed systems research and application, patterns have emerged about how we build these systems. We need a way to do locking of a variable, so that two nodes will not be able to write to that variable in a nondeterministic fashion. We need a way to do master election, so that if the master node dies, the other nodes can pick a new node to orchestrate the system.

Today, we use tools like etcd and Zookeeper to help us with master election and locking — and these tools have an onboarding cost.

Brendan illustrates this with the example of Zookeeper, which is used by both Hadoop and Kafka to do master election. Zookeeper takes significant time and effort to learn how to operate. During the construction of both Hadoop and Kafka, the founding engineers of those projects engineered their systems to work with Zookeeper to maintain a master node.

If I am writing a system to do distributed mapreduce, I would like to avoid thinking about node failures and race conditions. Brendan’s idea is to push those problems down into a standard library — so the next developer who comes along with a new idea for a multi-node application has an easier time.

Important meta-point: metaparticle is built with the assumption that you are on Kubernetes. This is a language-level abstraction that is built with an assumption about the underlying (distributed) operating system — which once again brings us back to standards. If everyone is on the same distributed operating system, we can make big assumptions about the downstream users of our projects.

This is why my mind is blown by Kubernetes. It is a layer of standards across a world of heterogeneous systems.

Serverless Workloads

Functions as a service (often called “serverless” functions) are a powerful, cheap abstraction that developers can use alongside Kubernetes, on top of Kubernetes, and in some cases instead of Kubernetes altogether.

Let’s quickly review the modern landscape of serverless applications, and then consider the relationship between serverless and Kubernetes.

Quick review on functions as a service:

Functions as a service are deployable functions that run without an addressable server.

Functions as a service deploy, execute, and scale with a single call by the developer. Until you make that call to the serverless function, your function is not running anywhere — so you are not using up resources other than the database that is storing your raw code. When you call a function as a service, your cluster will take care of scheduling and running that function.

You don’t have to worry about spinning up a new machine and monitoring that machine, and spinning the machine down once it becomes idle. You just tell the cluster that you want to run a function, and the cluster executes it and returns the result.

When you “deploy” a serverless function, the function code is not actually deployed. Your code sits in a database in plain text. When you call the function, your code is being taken out of the database entry, loaded onto a Docker container, and executed.

AWS Lambda pioneered this idea of a function-as-a-service in 2014. Since then, developers have been thinking of all kinds of use cases. For a comprehensive list of how developers are using serverless, there is a shared Google Doc created by the CNCF Serverless Working Group (34 pages at the time of this article).

From my conversations on Software Engineering Daily, these functions as a service have shown at least two clear applications:

• Compute that can scale up quickly and cheaply in response to bursty workloads (for example, Yubl’s social media scalability case study)
• Event-driven “glue code” with variable workload frequency (for example, an event sourcing model with a variety of database consumers)

To create a FaaS platform, a cloud provider provisions a cluster of Docker containers called invokers. Those invokers wait to get blobs of code scheduled onto them. When you make a request for your code to execute, there is a period of time that you have to wait for that code to be loaded onto an invoker and executed. This time spent waiting is the “cold start” problem.

The cold start problem is one of the tradeoffs that you make if you decide to run part of your application on FaaS. You don’t pay for the uptime of a server that isn’t doing any work — but when you want to call your function, you have to wait for your code to be scheduled onto an invoker.

On AWS there are invokers designated for whatever requests for AWS Lambda come in. On Microsoft Azure there are invokers designated for Azure Functions requests. On Google Cloud there are invokers reserved for Google Cloud Functions.

For most developers, using the function-as-a-service platforms from AWS, Microsoft, Google, or IBM will be fine. The costs are low, and the cold start problem is not problematic for most applications. But some developers will want to get costs even lower. Or they might want to write their own scheduler that defines how code gets scheduled onto invoker containers. These developers can roll their own serverless platform.

Open source FaaS projects such as Kubeless let you provision your own serverless cluster on top of Kubernetes. You can define your own pool of invokers. You can determine how containers are scheduled against jobs. You can decide on how to solve the cold start problem for your own cluster.

The open source FaaS for Kubernetes are just one type of resource scheduler. They are just a preview of other custom schedulers we will see on top of Kubernetes. Developers are always building new schedulers in order to build more efficient services on top of those schedulers.

So — what other types of schedulers will be built on top of Kubernetes? Well, as they say, the future is already here, but it’s only available as an AWS managed service.

AWS has a new service called Amazon Aurora Serverless — which is a database that scales storage and compute up and down automatically. From Jeff Barr’s post about AWS Serverless Aurora:

When you create an Aurora Database Instance, you choose the desired instance size and have the option to increase read throughput using read replicas. If your processing needs or your query rate changes you have the option to modify the instance size or to alter the number of read replicas as needed. This model works really well in an environment where the workload is predictable, with bounds on the request rate and processing requirement.

In some cases the workloads can be intermittent and/or unpredictable, with bursts of requests that might span just a few minutes or hours per day or per week. Flash sales, infrequent or one-time events, online gaming, reporting workloads (hourly or daily), dev/test, and brand-new applications all fit the bill. Arranging to have just the right amount of capacity can be a lot work; paying for it on steady-state basis might not be sensible.

Because storage and processing are separate, you can scale all the way down to zero and pay only for storage. I think this is really cool, and I expect it to lead to the creations of new kinds of instant-on, transient applications. Scaling happens in seconds, building upon a pool of “warm” resources that are raring to go and eager to serve your requests.

We are not surprised that AWS can build something like this, but it’s hard to imagine how it could emerge as an open source project — until Kubernetes. This is the type of system that random developers could build on top of Kubernetes.

If you wanted to build your own serverless database on top of Kubernetes, you’ve got a number of scheduling problems to solve. You need different resource tiers for networking, storage, logging, buffering, and caching. For each of those different resource tiers, you need to define how resources will get scheduled to scale up and down in response to demand.

Just like Kubeless offers a scheduler for the small bits of functional code, we might see other custom schedulers that people use as building blocks for bigger applications.

And once you actually build your serverless database, perhaps you could sell it on the Helm app store as a $99 one-time purchase.

Conclusion

I hope you have enjoyed this tour through some Kubernetes history and speculation about the future.

As 2018 begins, here are some of the areas we’d like to explore this year:

• How do people manage deployment of machine learning models on Kubernetes? We did a show with Matt Zeiler where he discussed this, and it sounded complicated.
• Is Kubernetes used for self-driving cars? If so, do you deploy one single cluster to manage the entire car?
• What does a Kubernetes IoT deployment look like? Does it make sense to run Kubernetes across a set of devices with intermittent network connections?
• What are the new infrastructure products and developer tools that will be built with Kubernetes? What are the new business opportunities?

Kubernetes is a fantastic tool for building modern application backends — but it is still just a tool.

If Kubernetes fulfills its mission, it will eventually fade into the background. There will come a time when we talk about Kubernetes like we talk about compilers or operating system kernels. Kubernetes will be lower level plumbing that is not in the purview of the average application developer.

But until then, we’ll continue to report on it.

The Gravity of Kubernetes was originally published in HackerNoon.com on Medium, where people are continuing the conversation by highlighting and responding to this story.

The New York Times makes most of its money off of subscriptions. Facebook makes its money off native advertising. Hacker News is funded by Y-Combinator. Each of these business models creates biases in the information that gets promoted on the respective platforms.

This is why I like to know the origin story and the business models behind the publications that I read. Published content is shaped by the profit motive of the publication.

And yet, last month, I repeatedly found myself reading high quality content on a Medium publication that I did not know the origin of: Hacker Noon. Hacker Noon is a popular publication that syndicates curated content written about software.

Let me explain “syndication”. Imagine that I just spent three days on a Medium post about functional programming, and I have zero followers on social media. How can I get people to read awesome post? The answer is syndication. I can submit my Medium post to Hacker Noon. This gives me free distribution, and it gives Hacker Noon free content — a win-win relationship.

But why was it worth it for Hacker Noon to spend time curating content? That syndication process takes time. You have to read through lots of submissions, sometimes you have to send it back to the author to have it edited. And this is all to build a following on Medium. I have not heard of Medium being a profitable platform to build a business.

It’s worth pointing out the difference between Medium and WordPress.

On WordPress, this model of curated syndication has worked to massive success — for example, the Huffington Post and TechCrunch. These businesses make millions of dollars from advertising networks, because they are built on WordPress, and WordPress is an open model.

A publisher on WordPress can install plugins that serve ads from third party providers like Outbrain and Taboola. A WordPress site can also install any kind of data collection scripts, to gather data on visitors, and sell it to the highest bidder.

The lack of third party plugins is the blessing and the curse of Medium.

Because there is no third party ecosystem, reading content on Medium is a beautiful experience. The page loads quickly and predictably. There are no random scripts that are blocking the page as they hog your browser’s resources. When you go to close the page, there is never a popup that asks you to subscribe to a newsletter.

When I read content on Medium, I am not getting slapped across the face with ads for reverse mortgages and açaí berries. I am not being tagged for retargeting. It’s a beautiful experience. But Medium seems like an ecosystem that would not allow for the content syndication business like Hacker Noon.

I wanted to know who was running Hacker Noon, how the business works, and what it says about Medium as a publishing platform.

Hacker Noon turns out to be part of a network of Medium publications called AMI. AMI’s network includes sites like Art + Marketing, Future Travel, and Fit Yourself Club — all of which are distinct syndication platforms. David Smooke is the CEO of AMI, and he joined me for an interview on Software Engineering Daily to explain how his business works, how he has scaled the content syndication business, and why he is betting on Medium. It was a detailed look into the state of online publishing and where it might be headed.

You can listen to the episode on iTunes, RSS, or from the browser.

If you don’t read Hacker Noon already, one article to start with that shows off the quality of content is Learn Blockchains By Building One. I interviewed the author of that article, Daniel Van Flymen, and it has been one of the most popular episodes of Software Engineering Daily.

In the 1960s, advertising agencies were high-dollar creative producers.

A client would come to an ad agency and pay millions of dollars for artistic messaging that would convince a consumer to buy a product.

How could you measure the success of these advertising campaigns? Maybe you could see success in the sales data. Maybe people were starting to talk about the product. Ultimately, success was defined by how satisfied the client was.

When it comes to measuring outcomes, advertising has always been a messy business.

Bob Hoffman’s long career in advertising included CEO positions at three different agencies. He helped huge brands craft their messaging, and grab consumer attention.

In Bob’s world of advertising, lots of money was spent on creativity. Were the campaigns successful? That depends who you ask.

In the old world of advertising, everyone acknowledged that success was subjective.

As human attention moved online, the world of advertising changed. Advertising began to move from TV and magazines to websites. Technology companies were formed to enable this new type of advertising–known as adtech.

These companies claimed to bring scientific accuracy to advertising campaigns. The biggest adtech player was Google, who perfected search advertising.

If you could imagine the opposite of what Bob Hoffman built his career doing, it might look like search advertising.

Bob’s campaigns were about creating a brand’s voice, with colorful art and subtlety and ambient messaging. Advertising was about turning a brand into an entity you recognize, teaching the consumer to associate Nike with fitness, or Dove soap with clean hands, or Cheetos with cheesy, salty attitude.

Search advertising, on the other hand, is just text.

You enter a search query, you are looking for black socks, and the top link that comes back is a line of text that says “cheap black socks.”

Search advertising catches people who have an intent to do something. They have stated their intent by typing into a box. With search advertising, a brand might not even need a sexy, flashy creative.

The idea of intent-based advertising was expanded with retargeting.

As you navigate through the internet, adtech companies are watching you, gathering data on your intent. Maybe you aren’t typing your desires explicitly into a search box–but you are clicking on articles and blog posts and tweets. With all your online interactions, adtech companies can figure out that you are looking for black socks whether you say so or not.

Money poured into adtech for very good reasons: intent-based marketing works. The shift to adtech put agencies in an uncomfortable position. If they couldn’t capture the advertising market by selling highly produced, unmeasurable creativity, they would have to make their money doing something else.

The situation was this: big brands like Procter and Gamble were buying most of their advertising through agencies. Procter and Gamble decided it wanted digital advertising.

The adtech companies were the ones who knew how to produce and distribute digital advertising. Since agencies had the relationships with the big brands, and adtech companies had the technology, agencies began to partner with adtech companies.

This is actually a simplified version of what happens. Agencies subcontract advertising deals to digital agencies. A digital agency buys technology from a slew of adtech companies: some technology tracks users around the internet, some technology places bids on advertising spots that will land in front of users. Because of all the middlemen, the incentives are aligned against the brands.

Contrast this world of agencies and adtech companies with the world of Google and Facebook.

They are a duopoly because they earned that position. By creating a single monolithic purchasing process, they have removed much of the risk that comes from a purchasing process stocked with middlemen.

But back to Bob Hoffman.

As money poured into adtech, user tracking, and Google, brands started to care more about metrics. When Bob met with a brand, the brand wouldn’t be asking about the cool new advertising campaign featuring a young actress drinking a Coca Cola. The brand would be asking about the click-through rate of a display advertising campaign.

Brands moved their focus to statistics, and away from creativity. And technology companies were happy to provide them with statistics. Whether those statistics were true or not is another story altogether. The industry was moving from creative BS to outright lying, and Bob decided to leave.

To hear my interview with Bob Hoffman, check it out on Software Engineering Daily.

We have done lots of reporting about advertising fraud for the last year, and it is a popular topic because people are often shocked to find that online advertising is inextricably linked to organized crime, surveillance, and Twitter botnets.

That’s not to say that online advertising doesn’t work–it certainly does! But understanding the dark underbelly of the Internet’s cash cow is a necessary precondition to finding a solution.

If you like this article, subscribe to the Software Engineering Daily Podcast on iTunes!

Take a look at the Venn diagram below:

If you like to blog and you’re looking to reach a bigger audience, consider podcasting. Not everyone has time to read blog posts, but everyone has time to listen to podcasts.

We all have a commute, have chores to do, and exercise. During these activities, people love to listen to podcasts.

By producing a podcast, you can reach the growing audience of people who consume hours of audio content daily.

Back in 2015, I was spending nearly four hours per day listening to podcasts. I listened while I was commuting to work, running, working out, cooking, or doing laundry. Having a great podcast made these activities so much more enjoyable.

But there was not enough good content for me. I wanted more material about technology, but I didn’t want “tech news” or op-ed podcasts . I wanted to hear from engineers about what they were building.

I started Software Engineering Daily because there weren’t enough technical, long-form software podcasts. Today there are 24,000 engineers who listen every day.

Still, there’s not enough good technical content. The world of software is expanding faster than anyone can cover it. Software is eating the world, and podcasts help with digestion.

That’s why this is the perfect time for you to start your own software podcast.

How to start your own software podcast

We are all new to software. The field is so young that new ideas are being discovered constantly.

Software is in everything we do these days. You can start a podcast at the intersection of software and anything you love.

Here are the basic steps to get going:

1. Choose a niche.

Do you like JavaScript? Blockchains? freeCodeCamp? Software has thousands of niches. Find one that you love to talk about.

At each of these niches, thought leaders are waiting to be interviewed. They are easy to find and contact through Twitter, Medium, and Quora.

2. Choose a style.

One-on-one interviews, group discussions, and storytelling can all be successful podcast styles. My article on different podcast formats includes some ideas for starting.

My personal favorite is the hour-long one-on-one interview. In an hour, you can dive deep into your guests’ beliefs about technology. They’ll say things that will surprise you and educate your audience.

There are many advantages to the interview podcast. It’s easy to find guests — if you invite someone to come on your show, that is a flattering proposition. The guests provide variation for the show, while the host provides some regularity.

It’s a successful recipe used by personalities like Jason Calacanis and Joe Rogan.

3. Prepare.

After you find your guest and schedule a time for the interview, it’s time to prepare.

Preparation is the most important difference between your podcast and all the other podcasts out there.

Write down one or two pages of short, concise questions for your guest. Try to order the questions in a way that form a narrative. Your outline creates the bones for your podcast episode. The interview itself adds on the flesh, organs, and skin.

Without bones, our bodies would be watery piles of mush. And without preparation, your interview will be a disorganized mess.

You will probably spend an hour or more writing down questions. You don’t need to ask these questions during the interview, but they are your safety net.

I’ve made outlines for each of the 750 podcasts I have done. A little preparation goes a long way. I always create a one or two page outline of questions. Here are some sample outlines from my show Software Engineering Daily.

After your preparation, you are ready to record the podcast!

“Hold on,” you might say. “Aren’t there a lot more things I need to do before I can podcast?”

Common technical questions about podcasting

Is it audio equipment for podcasting expensive?

No. An iPhone headset is fine to get started, and cheap USB mics are less than $20. In the early days of your podcast, most of your listeners will be your friends. They probably miss the days when you talked on the phone (since nobody talks on the phone any more).

Your first several podcasts are like big conference calls where most of the participants are just listening. It’s okay if the recording tools are no better than what you use for phone calls.

How do you record a podcast?

Zencastr is a free tool that makes it easy to record a podcast. It’s like a Google Hangout, but recorded.

With Zencastr, you create a new podcast, send your other podcasters a link, and hit record. When you’re done, you click stop and it uploads your file to Dropbox. If you want to be extra safe, have each person on the podcast record a backup file of their own voice.

How do you edit a podcast?

Basic editing is easy and free. Here are tutorials for GarageBand (Mac) and Audacity (everyone else). If you can afford it, spend $5 on a Fiverr editor.

Audio editing sounds scary to some people, but don’t let it intimidate you! If you are going to edit your own podcast, keep a text editor open during your show. Write down problems that you will need to edit later like, “20:00–20:23 Stacy was coughing.”

Where do you publish a podcast?

Use a publishing tool like Podsheets to give your podcast a website and put it on iTunes/Google Podcasts. After you record and edit your podcast episode, you will have a single audio file. A podcast publishing tool will let you create a post that goes along with the audio. These posts are what a podcast player on your phone shows to you.

To recap: setting up a podcast is easy and nearly free, assuming you already have a computer.

Podcasting is fast

After you set everything up, podcasting is a quick way to make content.

For example, if you add 60 minutes of preparation, 45 minutes of recording, and 75 minutes of editing, you get three hours of work for a 40 minute podcast.

If you hire an editor, you can get it down to less than two hours for a show.

In contrast, it takes me at least three hours to write a decent blog post. To write a long, quality blog post takes days.

Your first five podcasts will take longer than your next five. There are economies of scale to podcasting. The more you do, the smoother your process will become.

More benefits

Podcasting is public speaking in a controlled environment.

When you record a one-on-one interview podcast, there are only two people: you and the guest. If you say something you wish you hadn’t, edit it out — nobody else will ever hear it.

To recap, here are 5 reasons to start a podcast:

1. Build a brand. You can establish yourself as an expert after you begin publishing on a regular basis. You’ll gain street cred as an authority in whatever industry you’re focused on.
2. Form deep connections with listeners. A podcast is a way to connect with a highly-targeted audience. When people hear another person’s voice, a sense of trust and closeness forms.
3. Create content on your own schedule. You’ll begin to produce a consistent podcast once you get into the flow.
4. Go where the people are. People are busy. They’re driving to work or doing errands. They’re working out. They’re doing the dishes. They’re traveling. And unlike watching videos or reading blogs, they can listen to your podcast at the same time.
5. Become a better speaker. Podcasting lets you broadcast your message from the privacy of your home or studio, and you can re-record or make edits if needed. You’ll become more comfortable with recording and speaking over time. Podcasting will give you a chance to clarify your message and vision, and you’ll learn to present it in the best way.

If you’re still not sure whether you want to podcast, or have questions about how to get started, leave a comment on this post, or send me an email at jeff@podsheets.com.

If you like blogging, you should try podcasting. Here’s how to get started. was originally published in We’ve moved to freeCodeCamp.org/news on Medium, where people are continuing the conversation by highlighting and responding to this story.

Still aren’t sure why you should start a podcast? Check out Top 5 Reasons to Start a Podcast.

Once you decide to start a podcast, you need to decide how you will structure your show.

Here are some options.

The Interview

The classic podcast format. In each episode, the host interviews a different guest, diving into that guest’s expertise or their life.

There are many advantages to the interview podcast. It’s easy to find guests — if you invite someone to come on your show, that is a flattering proposition. The guests provide variation for the show, while the host provides some regularity.

It’s a great recipe, used successfully by personalities like Terry Gross, Ezra Klein, Joe Rogan, and Mark Maron.

The Host Discussion

Do you have a friend (or group of friends) who you have great conversations with every time you hang out? Every time you get coffee with them, you walk away saying “I wish I would have recorded that conversation!”

In the freewheeling discussion, 2–4 hosts talk about a different topic every episode.

Shows like Exponent and Pod Save America exemplify this format.

Sometimes these shows also bring on guests. The group of hosts interviews the guest, or the guest joins in on the freewheeling discussion.

The Highly Produced Narration

Highly produced narration takes audio recorded from several different contexts and mixes it together to tell a story.

Shows like This American Life, Serial, and Radiolab draw massive audiences due to their close attention to detail. They draw listeners in with polished storytelling.

If you love editing audio, the highly produced narration is for you.

There are several challenges to this format: you need different microphones for inside and outside your home studio. You need to move audio between multiple different devices. You need to have a vision for how your different recordings will fit together. These shows can take a long time to produce.

And you need an exceptional audio editor.

News

Podcasts are great for making sense of the news. I love to hear my favorite podcast hosts respond differently to the same piece of news, because it helps me understand that news through different perspectives.

There can never be too many news podcasts.

The Daily proved this by shooting to the top of the podcast charts due to its short length and excellent production.

News podcasts are easy to produce on a regular cadence, because there is always news. They are hard to produce, because you have to have something unique to say about the news.

Others

Podcasting is unexplored territory.

There are many other types of podcasts. Game shows, Game of Thrones analysis, monologues of people ranting, LeVar Burton reading books, fictional stories, concise explainers, music playlist podcasts…

Don’t be afraid to try something new. If you figure out a new podcast format, you might create a smash hit.

But also don’t be afraid to try one of the formats that is proven to work. Whatever podcast you decide to make will be inherently different than anything that came before it.

Whatever type of podcast you want to make, Podsheets can help you get started.

Let us know in the comments if you have any questions about how to get started.

Podcast Formats: Which One Is For You? was originally published in Podsheets Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.

The video prompted 12,000 orders in a two-day span after it was released, and has received over 24 million views as of February 2017.

In the video, the CEO of Dollar Shave Club walks around a warehouse doing funny stuff like tearing up paper, driving a forklift, and hanging out with someone in a bear costume. It’s great comedy, and cost nothing to produce.

The ad earned Dollar Shave Club venture investment from Kleiner Perkins and a16z. I personally started ordering Dollar Shave Club over all the other razor subscription services because I had seen this ad.

Without this ad, Dollar Shave Club’s brand might not have propelled it to a billion dollar exit in the face of strong competitors in a commodity market.

If viral advertising is so effective, why doesn’t everyone do it?

Many ad agencies believe that trying to make viral ads is like trying to catch lightning in a bottle. Like the “Mail Kimp” ad on Serial — didn’t MailChimp just get lucky that a 14 year old mispronounced MailChimp in an adorable way?

The reason you don’t see more viral advertisements is that ad agencies have largely given up on viral advertising. They believe it to be a game of chance.

And so the best viral ads usually come from in-house marketing teams, like Tom from Blendtec.

Viral ads can be created in a deterministic fashion, but most agencies are not well-equipped to deliver them. So the public narrative is that viral ads are simply something that happens randomly to a lucky few.

Brands hire ad agencies to create and distribute their ad content. The ad agency is accountable for delivering results.

If you are an account manager at an ad agency, would you rather

1. spend your entire budget trying to make a viral video ad OR
2. spend half of your budget on a bland video ad and spend the other half promoting it?

Today, most agencies choose option 2. If the ad doesn’t receive enough views, the agency can pay for bot traffic to shore up the numbers. The brand will receive a report documenting the number of ad views from the agency at the end of the transaction and they are none the wiser.

If you are trying to build a brand online, you are much better off crafting a viral advertisement than paying for programmatic advertising.

Originally published at www.quora.com.

Build a complicated, hulking four-month deliverable that nobody else commissioned and that nobody else will care about. Make a game, or model an elevator system, or an economic phenomenon. Put it on GitHub, but don’t expect anyone to look at it.You will learn so much, but that’s not where the most value comes from.

Building something big insulates your ego.

If you have built and shipped something cool and unique on your own, nobody can deny your identity as an engineer, even if you sometimes forget how to find all the subsets of an int array that sum to k.

This is so important — if you lose confidence that you actually belong at a strong company, you are dead in the water.

I’ve been where you’re at.

Spending months and months preparing for interviews, doggedly memorizing the cookbook answers for how to rotate a matrix, find anagrams, and so on. Cracking the Code Interview is an awesome book, but also reinforces the status quo of a certain kind of uninspired, assembly-line narrowness to the interview process.

It’s really sad that a lot of companies think this is the best way to screen people. I have to believe it’s not.

I’ve failed in more final round interviews than literally everyone else I can think of, and I bust my ass and write code after work and soak myself in engineering information.

But it’s OK.

Their loss. I’ve built stuff, and so many of the people I know who can close on these positions at FB, GOOG, etc have not. They are either scared to build from the ground up, or they continually start and can never push through to ship a project.

If I hadn’t built and shipped cool, difficult projects, I wouldn’t have this sense of validation and internal strength.

Originally published at www.quora.com.