This blog post is based on personal experiences, learned through lots of life lessons & hard work! If you want to hear specifics on a certain aspect of the marketplace, feel free to reach out to me on Twitter

Initially…

When I joined the company the objective had been to scale the API marketplace to infinity and beyond (!!). To become the largest single hub for API developers.

A lot of challenges and successful moments later, today the company works on the world’s most popular open source API gateway. Built in the open on Github as a developer-first product. They did really good in the microservice enterprise scene.

Working Marketplace Magic In A Developer-First Market

Hold on! What’s an API marketplace? What’s a marketplace business model?
In a marketplace you have:

• people selling some kind of goods (or services)
• people looking for these goods (or services)

As the provider of the marketplace you need to make sure sellers and buyers respect the rules of the marketplace and are satisfied with all aspect of the transactional experience.

These two groups need to be mutually interested in each other for the marketplace to thrive.

You, as in the entity who runs the marketplace, are trying to provide the best possible experience to both sides. The most old-school example of a two sided marketplace is eBay.
The more traditional marketplaces are:

• Amazon, Etsy, AliBaba

and some companies that deal directly in the b2c space, like:

• Uber
• Lyft
• Tinder
• Doordash
• Postmates

and many other..

A little background on how the API marketplace was born…

Around 2010 everyone started dreaming about connecting Apps with APIs.
The web had just stopped talking about widgets, mashups, HTML5 was at this stage *hot*, APIs were definitely the new cool thing.

Around 2011 the initial concept of an API marketplace was born. Initially as a mere index of public APIs — developers could push their API to the platform and provide documentation for others, “API consumers”, to learn and use it.

The early days

In the first iteration of the API marketplace, all you could do was track the amount of developers using an API, think of like ProductHunt for APIs 😸 — a list of people who would up-vote an API as they made request to it.

The more developers using an API, the highest the popularity of that API on the global marketplace index.

At this stage the marketplace’s sole service was authentication. You would make a request through the marketplace server with your API-key, you would get a token and then you could use this one-time (!!!) valid token to request any of the API’s hosted (or rather proxied) by the Marketplace. What a mess!

Golden API Adoption Days

Then suddenly around 2013–2014 everyone, literally, jumped on the API wagon, just as predicted. Hackathons were huge, people were living off pizza and beer and API meetups and conferences started popping up!

On Mashape things had to scale up on all fronts, as API usage would become more popular, and API requests too. Re-writing the marketplace and making everything work better was the core priority of the team.

Relaunching and completely rewriting the marketplace was a huge feat. The platform was reshipped as Mashape 5 with an improved design, a better user experience and more services for the end-users.
This came at a price, the engineering team spent countless nights sleepless after the launch to ensure a smooth transition.

Scaling the marketplace throughout the years took a HUGE amount of work. Luckily our team was young, full of passion and very energetic!
There were monitors held around the office focused on the KPIs. We noticed people were looking at the monitors when walking in but weren’t stopping by them as much as we would have liked it.
We then decided to send daily reports via email which we would discuss during our stand up meetings.

Looking at where users would get stuck and the feedback coming in through our customer support account had helped us immensely re-shape some rough edges of the marketplace diamond.

Changing/adding features that were requested by customers had to be prioritized (using a huge Trello board) properly. We had so much stuff we wanted to do. It quickly became our obsession.

We also focused on creating a lot of documentation, videos and FAQs for on-boarding.

This tough time have ultimately shaped the company — picking up pace, switching gears and moving into a leading microservices-first enterprise software position.

Scaling up the marketplace operations

Scaling up the marketplace required a lot of work: platform features, developer outreach, clever “startup-py hacks”, open source contribution and sweating. By the end of 2012 the platform only hosted around a thousand public APIs.

Meanwhile…

While other platforms like ProgrammableWeb were focusing on the indexing issue — this marketplace was focusing on adding useful services that would allow service providers to benefit from having their API on the Marketplace.

The idea was simple:
Create value for providers (sellers), the harder demographic to onboard and the consumers (buyers) will come.

As an API provider you needed things like:

• Caching
• Throttling
• Authentication
• Customer support
• Analytics
• Documentation

These tools would ensure your API would become useful to developers world-wide, but at the same time you had to focus on the quality of the results returned by your API.

API service providers learned the hard way that being developer-centric would set them aside from competitors and give them a bigger chunk of the market.

By providing these layers of services to API providers it meant that developers could spend more time writing good code and good documentation!

Throughout the years of 2013, 2014 and 2015 the marketplace reached critical mass, there were over 10000 APIs a large part of which were private and used by small businesses (digital agencies, small dev shops etc) that would sell access to their home-grown API to select third parties.

The benefit of using the Marketplace was that all the extra services were free, they would come out of the box — as a consumer you’d have the fastest way to find, use and implement APIs. As a provider you we were holding your hand to launch your API.

Dawn to Dusk In The Marketplace

After 2010… APIs exploded. That’s also what changed the marketplace.

APIs were driving businesses from garage startups to IPO-bound — Twilio had started just 2 years prior — similar companies as them, pioneers of API-driven businesses were pretty huge, already reached millions of dollars of annual revenue. SendGrid was another one.

These kind of businesses were getting huge and were never going to move to the Marketplace model.

⚡ Dissecting what could‘ve been done differently

Most if not all marketplaces follow this path of growth:

• They expand into new markets and locations
• They start to add new features, products for buyers and sellers
• They make it super simple to buy and sell API
• They grow and scale both on the supply and demand side

The API marketplace managed to do a lot of this, yet, not enough of it.

#1. Never Fail. Provide Options Instead.

Realize you are the biggest point of failure of someone’s stack.
Just like AWS/Cloudflare/Google, when their services go down everyone suffers. When the main API proxy went down, every single API would also suffer. In our world when your infrastructure powers thousands of applications and their customers (free and paid) getting even 10 minutes of downtime means everyone and their cousins will suffer. As a silly example think of weather APIs, traffic APIs, mapping APIs hosted on the marketplace powering mobile apps and more going down — think of the effect a little downtime produced on all the apps that rely on these.

#2. Trust and SLAs — Business Criticality

Businesses don’t mess around.
Big API consumers are large businesses. They have both internal and external APIs and as they scale, you scale.
The earlier reason, a marketplace being the biggest possible point of failure, means that you need to have a resilient API proxy. When your customer is a big enterprise you will have to guarantee a certain level of service. The best way to do that is to have the proxy deployed as close as your customer’s heart as possible. This was however not what the marketplace was all about.

Years later it became apparent that something had to change and centralizing things wasn’t going to work for everyone. Therefore with the creation of Kong this vision became more and more clear and eventually took over.

#3. Monetize Fast, Monetize A Large Demographic & Monetize’em Good

As an API marketplace owner you need to monetize all the services that you provide. There are many revenue models you can go after but the best one is providing a set of tools to the users of the marketplace and charging them a premium for access.
In the API marketplace a lot of the service were built and offered for free while they could’ve been monetized into chunks. Analytics, documentation, etc.
At the same time because of the way the API marketplace was built and because of problems #1 and #2, developers would never be able to trust a platform that handled billions of API calls. The demographic the marketplace was going after was too niche — small and medium groups of developers.

#4. Maintain & Update A Lot Of Community Code

Part of what made Mashape successful was the ability to consume API using own wrappers and libraries. This made consuming simple: A lot of copy paste and a few terminal commands and you would be consuming API like a boss. This clever trick alongside with the explosion in adoption of open-source libraries such as Unirest, available for most programming languages, made it so that anyone with little effort could consume an API from the marketplace. However as things grew and got out of control it became increasingly hard to maintain all the Unirest libraries, this is apparent by just visiting the “issues” section on Github that keep accumulating.

The future for API marketplaces

Today, there are a lot of new marketplaces that are developer focused.

Personally, I don’t think developer centric API marketplaces will take off among developers.

I love the idea that Zapier has built. The vision #3 (monetize the largest demographic — people who want to get sh!t done) and #4 (tools to scale) perfectly, while they’re probably still working hard on #1 and #2.

They truly delivered on the vision of 2-sided marketplace, not only developers (providers and consumers) but also non-developers!

That is:
Monetizing a platform that allows non-developers to use APIs (n.b. without technical expertise) in the tools they use every day. Monetizing API providers by allowing them to enter the marketplace to reach these non-developers.

Zapier has a lot of competitors such as StamPlay and Tray.io, clearly so they must be doing something right — especially across organizations that are pretty large and need to empower their marketing, operations and other teams that don’t necessarily write code.

In my opinion the reason the original API marketplace is no more, is a direct evolution of the demand of its own users. That is — with the creation of a open-sourced self-hosted software for developers there’s no need to centralize anything anymore. This allows any developer (or team of)to manage their own API on their own terms. As opposed to a big closed-source platform, you give the power back to the people!

It turns out it’s a winning strategy and as time has shown — API gateways are now a common thing.

Next time you start thinking to build a marketplace for developers, make sure you evaluate very carefully your business model — think far and further!

I love marketplaces. The concept is intriguing. I’ve spent the last couple of years working on 2-sided marketplace businesses. Heck I’m working on a marketplace business right now.

👉 Before you go: Connect with me on Twitter

Thanks for reading this far I had to dig through notes and memories — Now, if you made it this far you might as well just leave an emoji-reply or a text-reply below 👇! Or Clap 👏?

Lessons learned from building the best API marketplace in the world was originally published in The RESTful Web on Medium, where people are continuing the conversation by highlighting and responding to this story.

We last learnt how to make a collection that generates a playlist with music from your favorite artists using the Spotify API. We then ran the collection in the Postman App and Newman.

Today, we’re going to take this one step forward and emulate 2 awesome Spotify features for music discovery — Discover Weekly and Daily Mix. And we’ll use 2 new features in the Postman Pro arsenal to do this — Postman Monitors and the Postman Pro API.

Postman Monitors, launched in January 2017, lets you check for the performance, uptime and correctness of your API. It’s built on Postman Collections running on the cloud on a specified schedule.

The Postman Pro API exposes several endpoints to help you integrate Postman more deeply with your development toolchain. Using the API, you can create and update collections and environments, and run monitors, simultaneously syncing each action with your Postman app. All new features (e.g. Postman Mock Service) will be launched first via the API before making their way into the app.

As you would have guessed by now, we’ll use Postman Monitors to run our simple yet effective music discovery collections once a day (or week) on the cloud.

Quickly jotting down the workflow:

1. Add a new request to our collection which updates the environment at the end of the run using the Postman Pro API. This allows us to emulate the Persist Variables feature in the app.
2. Create a monitor with the updated collection and environment.

You can check out the documentation for the collection here and download the collection here

Create the Monitor

1. First, make sure you run the collection in the app with the new environment template. You’ll have to set up the environment as explained in the previous post.
2. Now head to Postman Monitors. If you’re not already on Postman Pro, you can start a free trial.
3. Create a new monitor selecting the downloaded collection and environment template. In the screenshot, I’ve set it to run every morning at 9AM.

4. Go to the details view, and you’ll see this screen

5. Click on the Run Now button (next to Edit Monitor) to check if everything works fine.
6. Finally, after a few manual or scheduled runs, your monitor results should look something like this!

Since we’re using the Postman Pro API to update the environment in the final request, we constantly expand our list of artists as shown below. Personally, I have discovered entirely new and obscure genres using this monitor!

I hope you found this series to be useful and an insight into the hard work we’re putting into making Postman Pro the most comprehensive toolchain for API developers. Stay tuned for similarly fun use cases of the Postman ecosystem.

“Daily Mix” with Postman Monitors and Spotify was originally published in The RESTful Web on Medium, where people are continuing the conversation by highlighting and responding to this story.

In this article I’ll briefly demonstrate how you can to make use of few community-driven tools which I came across recently and I definitely recommend: Typicode’s json-server and Rebilly’s Open API generator. By using a combination of them, you can quickly achieve a mock API server with really nice and useful documentation based in short deadlines.

The Open API generator: specification is first

I think it’s better to start with the specification as it answers many questions, even before making any real or fake data for the API.

The official documentation is sufficient to get started. There’s really no need for detailed tutorial here on how to use the tool in order to make a working project — just follow the steps. (And maybe push an empty commit on the gh-pages branch before first deploy just to make sure)

More importantly, you have to know why this generator is good and why I recommend it 😌

• It works with swagger editor out of the box, which feels like having a back-end for building your specification and documentation
• Your specification (aka swagger spec) can be split into several files, which makes the project more manageable and mainteanable
• Sleek documentation with ReDoc (though swagger-ui is also generated) I think ReDoc is excellent in following the API documentation trends.
• The results of your work can easily be deployed to github pages. It’s possible to make requests from here to external API server like the one we’ll have with json-server, as it supports CORS out of the box. Basically, as long as the server can accept requests from your github pages, these server really effectively as living documentation.
• Continuous integration with Travis comes out of the box. Every time you push an update in the specification on the master branch, Travis checks for errors and deploys to github pages if everything is ok.

When you follow the steps correctly, you’ll end up with a simplified version of RebillyAPI repository. That’s the final outcome that’ll motivate you to try the generator now, if you haven’t already done so 😊

Probably you might find other similar tools such as spectacle, however this tool worked really well for me and it covers everything I think one might need during a prototyping phase of an API project.

Thus, you have a tool to develop an Open API specification and deploy its documentation to github pages, making it nice and accessible for the public. With this Open API specification, you can later generate server/client code with Codegen, or just deploy the spec to services such as Cloud Endpoints or any other that will soon come with similar integration.

Basically, there are many opportunities available for you when you have the specification. Next step is to generate a server with mock data which matches the specification’s “contract” with the documentation or just any front-end app meant to work with the specification.

The JSON server generator: fake it till you make it

The JSON server tool is not really a generator per se, however it’s easy to integrate it as such in your toolchain. What I mean is, the tool is based on express.js and its popular middleware. You won’t need the express generator as you focus on the server. The endpoints are automatically discovered out of JSON files and you do not need express’ sophisticated templates views, routes, etc.

A simple function in a script run by a npm task can easily do for having a generator for the server part. Also, faker plays really well with this “generator” function. (no word play intended here) Check an example.

The main ideas you should understand with JSON server:

• It’s an easy start out of an example or existing JSON files
• It’s easy to make a generator script to work with the tool

Also, another benefit is that the server can be easily deployed. For example, if you decide to deploy on heroku, you’ll only need to update your server to use process.env.IP and process.env.PORT. The documentation points to the jsonplaceholder example, but surely hook.io or a similar cloud service will work just fine. Just keep it simple and working — the main idea of using this tool is to move fast and effectively.

Another option for generating a server on top of an existing specification is the swagger-server package which also integrates with express.js, however does not necessarily provide the ready filters, sorts, etc. that json-server does.

Additional ideas

I personally separated both tools in 2 repositories for the following reasons:

• The mock API server can be managed and deployed independently
• Data from the mock server can be easily fetched from the github pages documentation, as JSON Server comes with CORS enabled by default
• This same mock API server can be just thrown away when the real server comes into place to work with the documentation and/or application
• It’s also good to have the documentation repository separate to make it easier to manage in a similar way as the server. Good example is how both swagger ui, swagger editor and ReDoc are integrated into a single project focusing on developing specification and documentation

Further notes

That was a brief article sharing some love about open source projects which deserve an applause for being simple and effective helpers for developers.

If this article gets any positive feedback, I will make a follow-up story to this one showing few tips to improve the workflow of editing and publishing specification to documentation to a working API.

At any case, the search of more effective and well-integrated API prototyping, documentation and deployment continues.

If you thought this article was helpful, then consider recommending ❤ the story so more people will find it.

Prototyping APIs with Open API Specification and Node.js was originally published in The RESTful Web on Medium, where people are continuing the conversation by highlighting and responding to this story.

“Documentation is like sex; when it’s good, it’s very, very good, and when it’s bad, it’s better than nothing.” — Dick Brandon

The goal of this article is to inspire improved workflows for building and maintaining documentation for API-driven projects. In the end of the story, you will hopefully try out new approaches of documenting your code. As such, it automatically increases the value of your work through making it more maintainable and re-usable in less time and efforts.

Introduction

Swagger is one of the most popular and widely adopted specifications for RESTful APIs. In fact, the specification got off so well that numerous community tools are available. Also, the Open API initiative was born as a result of successful projects that use these tools basing on this specification in one way or another.

With the raise of popularity in 2016, leading experts started sharing advice on as speeding the development phase with Swagger and later making a good documentation of an existing API, again same spec being involved. These are few examples of well-written tutorials demonstrating benefits of having an API and a good toolchain, and documentation around it. There are more, of course, and I think this trend will stay solid for the next couple of years. That’s because the Open API specification makes sense just enough to empower developers and business people to have a common language of how an interface should work. It is as a contract of communication and data format between humans and machines.

In summary, the Swagger specification brings several clear advantages:

• We can write project specification in yaml or json formats. The yaml one is probably closer to humans, which is really nice. (Yes, there are UI programs for that as well ;)
• We can generate server and client code automatically, because when the specification is solid, the tools are solid too.
• We can also generate live documentation based on the same project specification. This is particularly useful and that’s why I’ll expand more on about this subject in this article.
• We can program in many languages and the result of our work will stay alive longer, because it conforms to a server <-> client “contract”. Meaning, as long as you can make a swagger specification out of you work, you can move from one technology stack to another.

Node.js is a popular choice for building the server part of applications, but it’s not the only one. The knowledge you can get from this article is transferable to almost any programming language.

What makes a good API documentation

Let’s now expand on the subject of having an effective documentation. There are many ways to document your software (and API), still, there are few major points which you have to cover to make it right. There is a common denominator of what makes one documentation better than another.

Auto-generation

I think that is the most important one. Writing documentation should take the least amount of time and efforts, and should minimize the maintenance.

You have to find a good way to write the least amount of textual explanations for your API, which yields the highest value of practical guidelines for the user of the documentation.

Include examples

I speak from my personal experience here — it’s really pleasing when the documentation does not make me think too much. My advice: include some sample request objects, or some code samples, etc.

As a consumer or a developer of the API, I want to have a quick-gratification access to something that I use and it just works directly. Rarely I want to open my advanced REST client or curl while reading API documentation because it takes off my attention and is basically forcing me to get out of the page I’m reading in order to have “the real feeling” about how something works.

Portable

That is a feature I consider equally important for an effective piece of documentation. This is when the auto-generated code can be either displayed as a page or an embedded window of information. It also means that the result of your documentation is re-usable.

If you are a JavaScript developer reading this, you will know that how fascinating the number of possible view frameworks solutions are. One day we have to make a simple page with Bootstrap, then with React.js, Angular.js, Vue.js, etc. No matter the framework or the tool, integrating documentation in an app has to be as easy as an instantiation. (In order to keep the spirit of high productivity and low maintenance costs mentioned in one of the previous points)

Swagger UI comes to the rescue

Swagger UI is a widely-adopted solution to the problems above. The only drawback being that the markup generated by the tool is not BEM-ish or anything like this, thus preventing front-end developers to make really nice themes on top of the solution. (although it’s partially going on)

When you look at the demo (or the editor), you will notice that there are similarities in the information presented.

Each endpoint of the API that is documented contains:

• Short description (the textual explanation we usually over-do)
• Example samples of request bodies, options to change request types, etc.
• Opportunity to try out the request <-> response operation

On top of that, swagger-ui documentation is auto-generated based on a simple yaml or json swagger specification file. That is super-nice, because the specification that a business person writes or configures through an UI can serve as both a contract AND live documentation!

We can do better than that

Yes! Even after following popular tools and tutorials, I felt there is room for improvement. Mainly — writing the documentation itself.

See, the problem of writing a single specification file and then using tools to generate the documentation is easy until the maintenance comes into play. Imagine that the yaml spec is maintained by the same developer or development team that is responsible for the API endpoints of the app server. Surely, there will be back and forth between the specification file and the code files that contain the actual implementation. It takes extra time and efforts to stay focused while jumping between editors and making sure that both resources are well-matched.

In my opinion, the process of “specification <-> development <-> documentation <-> implementation” can improve. Namely, it is possible to do all them at once.

How? By writing swagger specification in the documentation blocks of our code and plugging watch tasks in-between! Yes, the specification might still be in ownership of another one, not the developer. But even then — what if can just see live documentation based on the comments we’re placing in our code? Wouldn’t it be a motivating factor to write better in-code documentation? What is more, wouldn’t it be easier for developers to maintain both the code and the documentation of endpoints where the two are just next to each other?

Look at the following piece of code:

/**
 * @swagger
 * /login:
 *   post:
 *     description: Login to the application
 *     produces:
 *       - application/json
 *     parameters:
 *       - name: username
 *         description: Username to use for login.
 *         in: formData
 *         required: true
 *         type: string
 *       - name: password
 *         description: User's password.
 *         in: formData
 *         required: true
 *         type: string
 *     responses:
 *       200:
 *         description: login
 */
app.post('/login', (req, res) => {
  ... 
});

As you might have already guessed, this is a sample from swagger-jsdoc which takes the contents of the text after @swagger and parses the contents of the jsdoc documentation into a swagger specification. This specification can be then visualized, for example, like this:

It’s true that I made the screenshot from the built of the example, not from this tiny sample, but you get the point ;) (Continue reading and you will see much nicer interface of the same info)

Since I’m looking to improve this workflow even further (and make it more fun for developers to use the tool), I recently suggested a watch task in swagger-jsdoc command line tool that works just like this:

$ swagger-jsdoc -d swaggerDefinition.js routes.js routes2.js -w

In this case, after having a generated swagger.json the command line will continue listening for changes in routes.js and routes2.js and then re-generate the specification swagger.json on updates.

Also recently, the command line tool got a new feature to output yaml if the -o flag is used like this:

$ swagger-jsdoc -d swaggerDefinition.js apis.js -o swagger.yaml

In this case, apis.js will be searched for @swagger tags, but the parsed specification will be written to swagger.yaml file.

Having the swagger.json or swagger.yaml specification at this moment is a nice start. This file is the key re-usable resource that can be integrated with other tools such as swagger-ui which visualize the “soul” of the API in nice UI interfaces.

Improved API documentation workflow

Some time ago, an article about watching files changes with npm scripts got popular. I believe there is a reason why comments are not possible under the original article, but never mind, it got me on the path of researching the fs.watch() and its behavior. After experimentation I reached to prefer chokidar which is a module improving the fs.watch() interface. In fact, I liked chokidar so much that I took the opportunity to suggest a new feature in the CLI tool of swagger-jsdoc which is, duh, a watch task working with the API files. (Briefly mentioned in the previous chapter)

The chokidar module is also an integral part of the browser-sync module which, in turn, an already mature and battle-tested piece of software. It is widely adopted in various products and projects to increase development experience and productivity.

Having this understanding, I saw another opportunity — making a simple integration between instruments which integrate the chokidar module. I only needed a better understanding of Swagger UI which is the “glue”, or the “contract”, between the back-end server side (the API) and the front-end (client) side. I feel it’s comfortable (and smart) that the documentation pages are the first-hand client and consumer of the API. Also, I believe that this consumer should be easy to make and maintain, but still really nice-looking, so that there is something pleasant for our paying clients to see. With browser-sync Swagger UI got a bit more fun to work with.

In order to demonstrate the idea better in visual and kinesthetic way, I created a supplementary repository. It contains examples of the first approach that came to my mind keeping the KISS principle. Basically, there are few npm scripts that make the system work together.

First, you can have a look at the package.json manifest. Tasks are separated in two main categories: tasks for the server side (the API) and tasks related to the documentation. For the API development part, npm run api fires a server with nodemon and swagger-jsdoc As a result, any changes in the api/ folder trigger server restart and re-generation of swagger.json specification which goes to api-docs/ folder.

Secondly, the npm run docs:develop task fires a browser-sync server for the client side serving the api-docs/ folder which receives automatically updated swagger.json specification from the previous task. You can run those two tasks in separate terminal sessions or just create another parallel task following the examples of the package.json file with npm-run-all.

At this stage, you can freely update both the server and client sides of your project having near real-time feedback and results visible in the browser.

You might be wondering — why isn’t this the interface of Swagger UI I am used to seeing around! What happened? Well, I used a fork which looks nicer. Here’s an online demo which you can play with to get the feeling of what your documentation may look like after you enjoyably use swagger-jsdoc and browser-sync as shown in this repository.

The same workflow is possible with other file extensions such as .php just changing the input parameters and removing the binding to the nodemon server restart task, so that another standalone server is used instead. Here’s an example taken from a recent pull request which was included in swagger-jsdoc to parse swagger documentation to yaml output.

Deployment

I have intentionally left out the deployment part out of this story, because I believe there are many good options on the market and it really depends on the personal preferences how to manage this part of the process. For example, you can follow the scotch.io tutorial mentioned earlier to place the docs on heroku, or you can also include an npm script which works with gh-pages to deploy the assets directly to your repository, etc. The main thing is that the api-docs folder is already built and ready to be deployed as-is.

Myself, I plan to experiment with Google Endpoints in near future although it’s a new service. That’s because it supports the Open API specification and provides flexible options on deployment for both the API and the documentation on reasonable terms. An interesting point to notice is that the service guidelines show deployment only of the swagger/openapi specification file which can be generated by the swagger-jsdoc This is a good example of re-using single specification for multiple purposes.

Conclusions

Writing this article and sharing some code around it, I’m integrating tools that solve problems which have already been under the radar of the API Evangelist such as the need of better API documentation and UI deployment which was later followed up by findings about improved Swagger UI design solution. Few months ago, there was also a reflection article about the success of Swagger UI.

I hope that this article has inspired you to consider some new ways to improve your documentation workflows for your API project. I know that the idea of automated documentation generation with swagger tools is not new. Rather, I aim to introduce some new ideas on solving the problems which have been around on topic for a while with the tools available today to improve productivity.

The tools shown in the repository are free and vendor-neutral and the documentation build can be deployed to any place which serves you best. So, I encourage you to further experiment with the code and adapt it to work best for your specific needs.

If you thought this article was helpful, then consider recommending ❤ the story so more people will find it.

Agile documentation for your API-driven project was originally published in The RESTful Web on Medium, where people are continuing the conversation by highlighting and responding to this story.

Today, we are going to create our own on-demand playlist using the Spotify API and Postman Collections.

I love Spotify. Over the years it has helped me discover great music. Discover Weekly, in particular, has made me look forward to Monday mornings, with some amazing curated music delivered to the app. Usually, when I come across a great new artist, I look for their top songs and when a bit more adventurous, check out the related artists and repeat the process. Quickly jotting this down into a workflow:

1. Choose the artist(s) that you like.
2. Randomly choose one (let’s say The Who) from the list and GET the meta information using Spotify API
3. Get artists related to The Who and choose a random 5
4. Create a playlist named “The Who Mix”
5. GET the most popular songs from The Who, choose a random 5 and add to the playlist
6. For each related artist, do the same

And that’s it! We get a brand new playlist.

Click the Run in Postman button below to get the Collection and Environment in your Postman app.

You can check out the documentation for the collection here and download the collection here:

Setup the Environment

1. Spotify

a. Head to Spotify Developer and register, then create a new app in the My Applications section. Add the `client_id` and `client_secret` to your environment.
b. Under the newly created app config, add the following Redirect URI — “https://www.getpostman.com/oauth2/callback"
c. We’ll use the Authorization Code Flow to obtain the Refresh Token.
d. Open Postman, under a new request, click on the Authorization tab, select OAuth 2.0 and fill in these values:

Token Name: can be anything
Auth URL: https://accounts.spotify.com/authorize
Access Token URL: https://accounts.spotify.com/api/token
Client ID: {{client_id}}
Client Secret: {{client_secret}}
Scope: playlist-read-private playlist-modify-private
Grant Type: Authorization Code
Request access token locally: Checked

Click on Request Token, go through the OAuth flow, and add the refresh_token to your environment

2. Environment Config

Set the following variables in your environment
a. user_id: your spotify username
b. country_code: in ISO 3166–1 format
c. N: defaults to 5. Refer to the algorithm section for more details
d. artists: List of seed artists for the collection. You can find them as part of the Spotify artist link (Available in the app): e.g. for Oasis, the link looks something like https://open.spotify.com/artist/2DaxqgrOhkeH0fpeiQq2f4, choose the last section. The artists variable needs to be set up as an array of strings, that is, with square brackets and double quotes. So it’ll look something like: [“2DaxqgrOhkeH0fpeiQq2f4”, “2ooIqOf4X2uz4mMptXCtie”]

3. Run the Collection

a. Now run the collection in the Postman app using the Collection Runner

By keeping Persist Variables checked, the next time we run this collection, it will add the newly discovered Related Artists to the original seed and thus expanding the scope of discovery, each time we run this collection.

b. The same collection executes in Newman as well. Just export the Collection and Environment files and execute the following command

newman -c SpotifyPLGen.json.postman_collection -e SpotifyPlGen.json.postman_environment

In the final post of this series, we will schedule this collection to run every week for that true Discover Weekly experience.

Music Discovery with Spotify and Postman was originally published in The RESTful Web on Medium, where people are continuing the conversation by highlighting and responding to this story.

Traditional REST API are designed for when you want to allow a programmatical retrieve or post content, far from being optimal when you just want your application to be told when something changed. If that is the case, it would require polling at regular intervals, and that just doesn’t scale.

Polling an API is generally a wasteful and messy way of trying to retrieve that piece of information. Some events may only happen once in a while, so you have to figure out how often to make the polling calls, and you might miss it.

Don’t call us, we’ll call you!

Well webhooks are the answer. A webhook is a way for web service to provide other services with near real-time information using HTTP POST requests.

A webhook delivers data to other applications as it happens, meaning you get data immediately. This makes webhooks much more efficient for both the provider and the consumer, if your API doesn’t support them you should really do something about it (you hear me Salesforce?).

When it comes to designing webhooks there is nothing really close to a standard in the HTTP API literacy. Every service implements webhook differently leading to many different flavors of webhooks implementations out there.

After integrating APIs from more than 100 different services I can tell that the way a service exposes webhooks can be a deal breaker. So here are the things that make us happy when we need to integrate with a service exposing webhooks.

Self explanatory and consistent

A good webhook service should provide as much information as possible about the event that is being notified, as well as additional information for the client to act upon that event.

The POST request should contain a timestamp and webhook_id if one was given by the client when creating it. A type attribute should be included if you're providing different types of webhooks whether or not they are being sent to a single endpoint.

GitHub does that perfectly. Please don’t send just an ID that has to be resolved with another API request like Instagram or Eventbrite do.

If you think your payload is too heavy to be sent all at once give me the chance to make it lighter.

Stripe’s event types are a good example of this done well.

Allow consumers to define multiple URLs

When you build your webhooks you should think about the people on the other end of the wire that have to receive your data. Giving them the chance to subscribe to events under one single URL is not the best developer experience that you can offer. If I need to listen for the same event across different systems I’m going to end up in troubles and I need to put in between some Reflector.io type of thing. Clearbit please you have such good APIs, step up your webhook game accordingly.

Intercom does this very well giving you the chance to add multiple URLs and define for each one of them the events that you want to listen for.

UI based subscription vs API based subscription

Once the integration is in place how should we handle the creation of an actual subscription? Some services opted for a UI that guides you through the setup of a subscription some others built an API for that.

Slack killed it by supporting both.

It provides a slick UI that makes creating subscription very easy and exposing a solid Event API (which still doesn’t support as much events as their Real Time Messaging API does but I’m sure they’ re working on it).

An important thing to keep in mind when choosing whether or not providing an API for your webhooks is at what scale and granularity your subscriptions are going be available and who is going to configure them.

I find it curious that a tool like MailChimp forces a non technical audience to mess with webhooks configurations. By making webhooks available via API, any third-service (e.g. Stamplay, Zapier or IFTTT) that has a Mailchimp integration could just make that happen programmatically and build better user experiences.

For creating new webhooks subscriptions via API, you should treat the subscription like any other resource in an HTTP API.

A very good one we’ve been working with recently is the completely renewed webhook implementation made by the Box team which was released this summer.

Securing webhooks

Once someone configured his service to receive payloads from your webhook, it’ll listen for any payload sent to the endpoint.

If consumer’s application exposes sensitive data, it can (optionally) verify that requests are generated by your service and not a third-party pretending to be you. This isn’t required, but offers an additional layer of verification.

There are a bunch of ways to go about this — if you want to put the burden on the consumer side, you could opt to give him a whitelist requests from IP address — but a far easier method is to set up a secret token and validate the information.

This can be done at different degrees of complexity starting from a plain text shared secret like Slack or Facebook do

To more complex implementations. As an example Mandrill signs webhook requests including an additional HTTP header with webhook POST requests, X-Mandrill-Signature, which will contain the signature for the request. To verify a webhook request, generate a signature using the same key that Mandrill uses and compare that to the value of the X-Mandrill-Signature header.

Subscriptions with expiration date

The odds to face an integration of a service that exposes subscriptions with an expiration date is not very high today but we can see this becoming a more common feature. Microsoft Graph API is an example. Any subscription you perform over API expires 72hours later unless you renew it.

From a data provider standpoint it makes sense. You do not want to keep sending out POST requests to services that could be no longer up and running or interested in your data but for all those who are actually interested it’s an unpleasant surprise. You’re Microsoft: if you can’t afford the heavy lifting who is supposed to do it?

Conclusions

The webhook landspace is still fragmented but common patterns are eventually coming up tough.

At Stamplay API integration is a thing. We face the integration challenges on a daily basis and OpenAPI specifications like Swagger, RAML or API Blueprint can’t help because none of them supports webhook scenarios.

So if you’re thinking about implementing webhooks I invite you to think about their consumption and look at examples like GitHub, Stripe, Intercom and Slack API.

PS. Medium any plans for webhooks? Come on, RSS feeds are so old school.

Update: Medium actually does provide a way to get realtime notifications through http://medium.superfeedr.com/ 👌

Webhooks do’s and dont’s: what we learned after integrating +100 APIs was originally published in The RESTful Web on Medium, where people are continuing the conversation by highlighting and responding to this story.

Introductions to languages or software systems usually start with a “Hello World” exercise but most APIs are reference manuals only useful for experts. The docs are the first step in a beginner’s learning curve. However with API documentation the developer usually sees a long list of API endpoints in one single page that makes it really hard to get started with the API. It is my belief that API documentation should be similar to learning programming — explaining basic concepts with specific examples. As someone builds up their skill levels, they can use code to solve more complex real world problems.

How do you go about building a great first user experience? Being Postman, we’ve been exposed to several ways in which API publishers approached this problem. Below is a list of questions to help you think through the process of making your API simpler to understand and faster to use.

In part 2 of this post, we’ll talk about our approach at Postman to solving some of these problems with examples and a more in-depth analysis of specific documentation.

Step 1. Who is your user?

Users for your API documentation will come from different backgrounds. They might be backend developers, frontend developers, testers or architects. They’ll know different programming languages and different frameworks. Talking in their “language” will help you get your point across. But everyone’s language is different. Navigating the paradox is critical. Having a range of examples in different programming languages is important — don’t assume everyone knows Java and XML.

Step 2. What does your API do?

Is this API itself the product? Or is this a way for consuming/storing information created in your app? Explaining how your API can make someone or something better will help set an anchor point. At this point it is critical to describe what your API does NOT let someone do saving the user a lot of time and frustration.

Step 3. How does one access your API?

This is where a lot of users get stuck. Getting an authorization token easily for the appropriate access level will help someone try out the API quickly. You can do this by generating an access token automatically for a user if he is signed into your developer portal or your website. You can also create a token management interface similar to Github that can provide users with both flexibility and security. You should have a separate section for authorization against production use cases and how to properly secure a service.

Step 4. How can someone quickly use your API?

Draw out common use cases and “Hello MyAPI” samples. You might solve the majority of problems for your first user right here. This might be a basic how-to on getting and fetching data, on combining different end-points or even a cool use case that might be buried inside your reference docs. Knowing your user (See point 1 above) will help a lot.

Step 5. How can someone integrate your API?

Your API will be used with other internal or public APIs. Examples of how to combine your API with other data sources, data elements to look for inside JSON bodies to transfer data, or other common workflows that your users might want to be aware of.

Step 6. What is the extent of your API?

Finally, write reference documentation for your API. API references should describe possible request and response types — not just the most common one. Sample code in the user’s choice of language will help him run the API docs quickly.

Step 7. Where does someone ask for help?

What happens when things don’t work as expected? Where does the user ask for help? Linking to an FAQ, your support channel or your community page will help the user get unstuck faster.

Conclusion

API documentation is a complex undertaking. Thinking about your user’s first experience of your documentation will help them get started faster and your API’s value more apparent. Creating reference documentation is great but there are several components that I encourage you to consider.

References

• Teach Yourself Programming in Ten Years is a great reference that talks about learning by doing
• What to Write says that a well-documented project needs to have different forms of documentation

Originally published at blog.getpostman.com on April 25, 2016.

Simplifying API documentation for a great first user experience was originally published in The RESTful Web on Medium, where people are continuing the conversation by highlighting and responding to this story.

Okay, so, it’s 2016 and you don’t know what REST is, what an API is, or how to design an API with this magical thing called REST that everyone keeps talking about. You’re frustrated by the phrase “Just Use REST,” because when you go to find resources on the topic, they’re hard to discover if you don’t know what to search on Google.

I’ll start this off with a short story. I graduated from university and somehow still had no idea what an API was. It wasn’t because I was lazy and never learned it, but more so the fact that it’s not really something that was talked about often at school. Then, I went looking for jobs, and all of the postings started talking about Web APIs, REST, and that I should have an understanding of it.

Turns out, this is a foundational concept that many jobs in tech require their staff to understand now. So why did I never learn it in school? I don’t really know (I really don’t have an answer, but it seems Comp Sci programs struggle to keep up with the latest tech like boot camps do), but needless to say I set out to learn all about RESTful APIs. I now want to pass that knowledge on to anyone who finds themselves in a similar situation.

I graduated from university and somehow still had no idea what an API was.

The goal, ultimately, of this mini-guide is to help you get up to speed with an industry best practice that you should know.

What This Guide Is Not:

A fully fledged, granular guide on REST and APIs. That would just require so much detail, and probably not the most suited to a series of Medium articles.

What This Guide Is:

A get-up-to-speed guide on REST and APIs…the “essentials,” if you will, to modern API design in 2016. I will explain a lot of details, and if you want to learn more, I will provide my sources for what I would approve as excellent next steps in the RESTful API learning process.

Structure of the Article:

• Glossary of Terms
• What is an API, and what is REST?
• Common Conventions / RESTful Principles
• REST Interface Constraints
• Summary So Far

Glossary of Terms

Before we really get started, I want to supply you with a few key terms I’ll be using in the rest of the article. This will help you understand what I mean when I mention certain concepts.

1. Hypermedia: As described by Wikipedia, Hypermedia “is a nonlinear medium of information which includes graphics, audio, video, plain text and hyperlinks.”
2. Resource: In RESTful APIs, resources are identifiers to any piece of interesting information the server can provide. For example, a resource could be the users of a system, a user’s timeline of posts, or a blog post, like this one. A practical example is twitter.com/Stuart_Weir, where my name is the identifier of a user resource in Twitter’s servers.
3. URI / URL: URI stands for Uniform Resource Identifier, which is simply a set of characters that is used to identify a resource. As we saw above, twitter.com/Stuart_Weir is a resource, but this resource is identified by a URI. A URL, or Uniform Resource Locator, is the most commonly used version of a URI, and is often called a web address. The whole of twitter.com/Stuart_Weir is the URL.
4. State: this is a pretty straight-forward term, but quite simply, it’s used to explain the current “state” or “context” of an application. In RESTful APIs, Hypermedia (see #1) is used as the engine of application state.
5. Media Types and Content-Type: A Media Type is the specified data format that will be sent in the Request / Response of a client-server exchange. A Content-Type is the HTTP Header field used to specify the Media Type. For example, Content-Type: application/json would be in the HTTP Header field, that specifies a Media Type of application/json.
6. Request / Response: A Request / Response is an interaction between the client and the server. The client will send a request to the server to do something, or get some data, and will wait for a response from the server on the status of that request, along with any extra data pertaining to that request.

To sum all this up, let’s quickly use all these terms in an example. A client (my web browser), sends a request to the URL www.twitter.com, specifying a Media Type in the HTTP Header field, “Content-Type.” Twitter responds to my request with an HTTP status specifying that everything went okay, and the data my browser requested (the web page). From this web page, I have a few options. I can post a new tweet, I can go to my profile, etc. Let’s say I want to go to my profile page. I click on my profile, and this sends another request from my browser (the client) to the server with the Hypermedia link www.twitter.com/Stuart_Weir. Notice that my request was for twitter.com, with the /Stuart_Weir resource appended. Twitter’s server sees that “yes, I have this resource,” and sends it back to my web browser, where I see my profile page.

For a more full-fledged glossary of API terminology, I highly recommend taking a look at Mashape’s API Glossary. This is an incredibly useful resource for reference, and created by what I like to call the “King Kong of APIs,” Mashape.

Now that we have a bit of foundation laid out in terms of terminology, let’s get to the heart of this article.

What is an API, and what is REST?

An API is an Application Programming Interface. This sounds super technical, but it actually really isn’t. It’s essentially an interface for programmers, much like a User Interface (UI) is an interface for product users.

To give a quick example, when someone directs their browser to twitter.com, it takes them to a UI that allows them to do a few things. Twitter will ask them to log in or sign up, and from there the UI will change/update so that the user can do even more things with the service.

Much like the Twitter example, APIs are essentially a more technical version of this. RESTful APIs expose resources (see above) that a programmer can interact with. APIs (and RESTful APIs more specifically) should be easy to use and easy to discover. In other words, like when a user goes to Twitter, the user should understand how to use Twitter, and discover different features of Twitter easily. In the same way, great RESTful APIs allow programmers to easily use their features, and discover the API quickly.

REpresentational State Transfer (REST) offers a way to structure APIs and offers certain opinions and conventions that make building APIs easier, more efficient, and standard, so that a client user (a programmer wishing to use that API), can quickly get up to speed with the API simply by understanding it follows certain conventions. REST was first introduced in Roy Fielding’s dissertation, Architectural Styles and the Design of Network-based Software Architectures. For a full overview of REST as fielding defined it, please check it out here.

REST is a stateless, client-server architecture that offers a cache-able protocol, and it’s almost always implemented via HTTP. It typically uses HTTP methods for Create, Read, Update, and Delete (CRUD) operations, and as such, is standards based. We’ll get into the definition of some more of this terminology later. REST is also language independent, meaning it can be implemented in numerous programming languages, so a REST API server built in Ruby, for example, can be consumed by a Go(lang) client. If we’re to follow with this Ruby and Go example, you may be wondering, “how does data written with Ruby code transfer to data that Go can understand?” Good question, but I’ll get to that in a later post.

Okay, so what else is there to know about REST? For now, let’s quickly sum up what we just learned about APIs and REST. An API is an Application Programming Interface, which is essentially software that exposes some functionality that other software can consume. In RESTful APIs, this is built off of a client-server architecture that we’ll take a look at in more detail in a moment. REST allows for us to transfer the representational state of a resource from the client to the server, and vice versa. Moving on.

RESTful Principles:

REST comes with a common set of conventions and principles in order to architect powerful, flexible, and scalable APIs. For example, one common convention with REST is that when handling a URL’s method (GET, PUT, POST, DELETE, etc), we use nouns instead of verbs for the resource name. This means that www.example.com/getusers would be www.example.com/users, and that will GET the users resource. Here are some more:

1. Client-server architecture: this is to help separate concerns, as managing state between a client and server would become exhausting, not only for a programmer, but exhausting on the system as a whole. In the example above, the Ruby API was the server, and there was a Go client.
2. Communication should be stateless: the interaction between the client and the server should be without state, meaning each request from the client to the server is a self-contained interaction, that does not require context from previous requests. However, both the client and server can hold their own state.
3. Cache-able: being able to cache a response from a server can greatly speed up interactions, in some cases, completely eliminating them.
4. Uniform interfaces between components: this is a distinguishing feature of REST, where the independent evolution of components is encouraged. This is because the actual implementation of the components (the code) and the services (what that code provides) are decoupled. In the Ruby and Go example, the two components were the Ruby RESTful API and the Go client, whereby the implementation of the Ruby code is unimportant to the Go client, as long as the Go client receives the Media Type it expected to receive, among others.

REST Interface Constraints:

Okay, so we now know some of the core principles behind what makes RESTful APIs RESTful, but what constraints do we need to implement on our RESTful APIs? Here are the main four:

1. Identification of Resources: One of REST’s key abstractions is a resource. These are identified in requests, such as www.example.com/users, where the resource in this case is users. This is conceptually separate from representation of the response, which could be a Media Type specified by text/html or application/json, for example. More explanation in #2.
2. Manipulation of Resources Through Representations: A representation can indicate the current or desired state of a requested resource, as well as other forms such as error conditions for a response. The data format of a representation is known as a media type, as explained above in the glossary section. If a client holds a representation of a resource, such as a representation of all the users, it has enough information to modify or delete this resource. So, if a request for the resource users is sent from our Go client to the Ruby API, a representation of that resource could be the response. Once the Go client has a representation of that resource, it might execute a DELETE request on all the users, or a single user resource if it has the authorization to do so. Be careful not to confuse a resource from it’s representation. Think of it as a box filled with old toys you had as a kid: The box has a name written on it, “old toys,” and inside the box is the representation (the toys) of that resource (the “old toys” box). This isn’t entirely accurate, but hopefully it helps paint an image in your head.
3. Self-Descriptive Messages: A message should contain enough information to describe how to process it. For example, if the response to a GET request is JSON, the Media Type should be specified as application/json in order for the response to be properly processed. There are a few things going on here. In the GET request, the request specifies that the HTTP method is GET, and the Media Type is application/json. So, inside this message, it’s self-descriptive because it specifies what to do with the data it’s requesting. This is why the message is also referred to as stateless or context-free; the message requires nothing outside the scope of itself to be processed.
4. Hypermedia as the Engine of Application State (HATEOAS): This is a fancy way of stating that what actions are available to the client vary as the state of the resource varies. In RESTful APIs, we use Hypermedia as the way we move state around. Hypermedia often is a link to another resource, however it can also be audio, video, graphics, or plain text. As described in this article, “a RESTful application enables the server to inform the client of the possible ways to change the state of the application via hypermedia.” To use the Twitter example again, say a user navigates to twitter.com. The user has two options: log in or create an account. These are both controlled by hypermedia links, which are used as the engine of application state. Say the user doesn’t have an account. They fill in their details, and hit “Submit.” A POST request is sent by their browser to the server, which then changes the application state of Twitter (to one where the user is now logged in). This is an example of using HATEOAS.

To sum up some of the concepts of REST covered so far, here’s what Raphael Simon, one of creators of goa had to say about REST:

REST services expose state through resources (state can be data in a database for example). Each resource represents an abstraction that usually maps to a model in the backend (a “customer”, a “product”, etc). It doesn’t have to be a one to one though, for example relationships may be exposed as resources as well or may be hidden from the API depending on what the service needs enabled. Resources are managed by making requests to the service that exposes them. In HTTP REST establishes the following mapping:
* POST to the collection of resources to create a new resource
* PUT/PATCH to an element in the collection to update it. “Usually” PUT is used for full updates while PATCH is used for partial updates.
* GET to the collection returns all resources
* GET to an element of the collection returns that element
* DELETE to an element of the collection deletes that element

From a strict REST point of view, this is pretty much it. In reality APIs tend to expose additional actions on resources. REST dictates that these should be represented as resources. For example if a service exposes a “vehicle” resource and the intent is to allow the API user to start the vehicle, REST would dictate that a “started_vehicle” resource be introduced for example. Creating that resource would return a started vehicle. Sometimes though that is unpractical (typically actions that tend to update state like the example here). In these cases resources define additional actions typically located under the resource location.

To Sum Up What We’ve Covered…

I covered a lot of content in this post, so feel free to go back, read it (or parts of it) again. It’s okay to not fully understand what’s going on yet. However, what we’ve covered so far is this:

1. Application Programming Interfaces (APIs) are like User Interfaces (UI) but for programmers. They should be designed to be easy to use and discover, much like UIs.
2. REST stands for REpresentational State Transfer. It was initially proposed by Roy Fielding, and provides a set of conventions that standardizes the creation of uniform interfaces in APIs.
3. REST is based on a client-server architecture, is stateless, and is preferably cache-able. Most often it is implemented via HTTP as to use HTTP’s methods for CRUD operations.
4. REST’s API constraints include the identification of resources, the manipulation of resources through representations, should have self-descriptive messages, and Hypermedia is the Engine of Application State, meaning that the actions that are available to the client is based off the state of that resource.

What’s Next?

Next up we’re going to take a look at some main considerations that go into designing a RESTful API, building off of our current working knowledge so far. Then, in the third and last article, we’ll put it all together by implementing that design in code.

Stay tuned for more in the coming weeks!

Sources:

• REST API Tutorial
• Choose an HTTP Status Code
• API Glossary
• Ideal REST API Design
• Awesome API
• JSON API
• JSON HAL
• On Choosing a Hypermedia Type for your API
• How REST Constraints Affect API Design
• REST - The Short Version

Objects at REST, Part 1: What Is It? was originally published in The RESTful Web on Medium, where people are continuing the conversation by highlighting and responding to this story.

This latest release builds upon our existing Collection Runner. It introduces a new function called postman.setNextRequest. As is evident from the name, you just need to specify the name of the subsequent request and the runner will take care of the rest. It has the following signature:

postman.setNextRequest(“Request Name”);

Let’s go over an example workflow so you can get a sense of how this works.

You might have met Postman’s Chief Happiness Officer earlier.

This is Cooper. He’s one year old and eats three meals a day. He’s allowed to eat only six pre-approved food items. We store them in a database. Cooper is very picky. You can’t repeat an item on the same day unless you want to risk a tantrum. So I want to generate a random meal plan for him at the beginning of the week.

The workflow:

First makes the API call to GET the list of approved food for Cooper from the database.

Then it creates an entry for each day, 3 times a day, 7 days a week

All 3 meals in a single day have to be unique.

If the API call fails, poor Cooper goes hungry 😟

You can check out the documentation for the collection here and download the collection here:

In case you’re curious, I ran the collection and here’s what Cooper’s eating this week!

In general, the postman.setNextRequest parameter is associated with the request and can be set in the pre-request or the test script. In case of more than one assignment, the last set value is considered. If postman.setNextRequest is absent, the runner defaults to the linear execution and moves to the next request. postman.setNextRequest(null) terminates execution.

I hope you find this useful. postman.setNextRequest is just the beginning and we will push out more APIs to easily create complex workflows in the coming quarters within Postman. Stay tuned!

Conditional Workflows in Postman was originally published in The RESTful Web on Medium, where people are continuing the conversation by highlighting and responding to this story.

Earlier this year I joined Mashape, an API platform and marketplace allowing you to distribute, monitor, and manage your APIs. While I was familiar with what an API was, I had never made one myself. I learnt programming when I was in college, and had some experience with Python at work (mainly scripting). Over the past couple of weekends, I took it as a personal challenge to learn how to create my own API, as well as experience some “real-world” coding. I worked on what I call the “Smarthome API”, an API for home automation that can control the devices in my house, such as the lights, fan, WeMo, door lock, and climate (via Nest).

In this post, I have documented my journey, the steps I have taken, the learning, and experiences along the way (mostly of my housemates who had to live in darkness while I was testing the app).

1. Defining the problem

My housemate recently installed several flavors of home automation hardware:

1. Z-wave switches and locks that enable home automation over a wireless network. We use Vera, a smart home gateway, to interact with the switches and devices.
2. A Nest thermostat, of which we use the mobile app to adjust the temperatures and switch between “Home” and “Away” modes.
3. Belkin WeMo appliances.

All these products turned out great when we found ourselves too lazy to get out of bed to turn off the light switch, or check if the door is locked. However, it was rather cumbersome to manage different apps, or to manage every device individually.

I wanted to have more flexibility and control over these devices. For a start, I scoped out these basic requirements for my API:

• Be able to retrieve the list of all the switches/ devices (lights, locks, Nest) in the house — GET method
• Be able to retrieve information on each individual switch/ device, e.g. Name, current state, etc — GET method
• Be able to modify the state of each switch/ device, e.g. turn a light on or off — PUT method

2. Choosing a language and framework

This was easy. Since I was most familiar with Python, I decided to go with Flask (a Python micro-framework), because it is lightweight. Of course, there were many other options, but I’ll not be going into details on the pros and cons of each.

PS. For the rest of this post, I’ll be referring to the switches/ devices as Lights. In reality, they control not just the lights, but also the fan, WeMo, etc. “Lock” refers to the door locks, while “Nest” refers to the climate control.

3. Building the scaffolding / creating the models

I created the model class for a Light object (on hindsight, I probably should have named it “DeviceSwitch” instead) with 4 properties:

• id — device id as specified by Vera
• name — name of the device/ switch to be controlled
• room — name of the room which the device/ switch is located in
• status — 0 for when the device/switch is off, 1 for when it is on

Next, I created my first endpoint: GET /lights which was supposed to return a JSON with information of all the Lights. I fed it with some dummy data, and saw that my method was working correctly.

The second endpoint: GET /lights/<id> involved returning a JSON with properties of the requested light. Once again, the method was verified to work correctly with mock data.

Changing the state of the Light was slightly more complicated, as the PUT request would need to take in an <application/json> content type with the desired state. For this, I made use of the Postman Chrome App which was able to verify that my method was working correctly. Success!

At this point, I pretty much went around the house and showed off my wonderful “working” API to my housemates, although it didn’t exactly do anything. They were not impressed ☹

4. Figuring out the Vera interface

The difficult part came when I had to figure out how to make requests to Vera to actually control the hardware. I opened the Vera control panel and used Chrome’s inspector to filter the data packets that were sent and received while turning lights on and off.

For a start, I figured out the request URL which Vera uses to retrieve all the devices.

It turned out that the request URL followed a simple pattern, and the I could request device information with just two lines of code:

p = {‘rand’: random.random()}
response = requests.get(“http://192.168.1.88/port_3480/data_request?id=user_data", params = p)

I used the same method to figure out the requests that I needed to make to Vera in order to change the state of a Light switch.

All that was required was the id of the device, the target state to change to, as well as a random number parameter.

p = {‘DeviceNum’: id, ‘newTargetValue’: <the new state>, ‘rand’: random.random() }
response = requests.get(“http://192.168.1.88/port_3480/data_request?id=lu_action&output_format=json&serviceId=urn:upnp-org:serviceId:SwitchPower1&action=SetTarget", params = p)

On hindsight, this was much easier to explain, but it took me some effort to learn and understand exactly what I was doing then.

5. Making it work

Since I had figured out how the HTTP requests worked, all that was left was making my API perform the desired action and return the correct response. (Yes, my API was still not really “working” after all the work ☹)

To list the lights in my home, I inspected the JSON response from the HTTP request, retrieving the values of id, name, room, and state of the device. I parsed the variables into a newly created Light object. Iterating through the entire response, I was able to build a list of Light objects which were then returned by the GET /lights method as a JSON.

To switch the state of a light, I decided to have my API accept a PUT request with the id in the URL parameters and the requested state in the JSON body. Implementing this was a 2 step process:

1. Verifying the legitimacy of the JSON request
2. Performing the actual request to Vera and returning a legit response

For a start, I only verified that the JSON request defined a proper state which could be used to control the Light device. If the state was specified, I’d send the request to switch the state. If there was no error, the method would return an OK response. Otherwise, it should return the error.

Although my code was somewhat messy and ugly, it worked. And I made sure everyone in the house knew it by randomly switching on and off the lights in their room ;)

6. Doing the same thing for other devices

Once the three methods worked for Lights, it was easy to replicate them for the Locks. Most of the code was pretty straightforward and looked pretty much the same, except for some minor URL changes in the Vera HTTP request. However, I did enforce that a password needed to be parsed in the JSON request when changing the status of the lock. (I definitely didn’t want random hackers being able to remotely lock and unlock my front door!)

7. Refactoring the code

Getting to this point was a proud moment for me. However, my code was messy and ugly, with inconsistent variable names, multiple nested loops and conditions, and a ton of duplicated code. One key issue was that my JSON response was returning the room id instead of the actual room name. I needed to retrieve the room names from the Vera API once again, and return them in my response.

8. Adding verification functions and increasing sleep time

At this point my API worked for switching individual devices on and off. I tried to create an endpoint that would turn all the lights off. However, I found that not all the lights were being turned off! Upon investigation, I realized that all my requests to the Vera API was succeeding and returning an OK response even though not all the lights were turning off in reality ☹

While I grappled with the issue, my unfortunate housemates had to suffer in darkness, since I refused to switch on the lights for them until I fixed the bug.

To resolve the issue, I decided to add a verification function to ensure that the device state switch was successful. Otherwise, I would send the request again. I also added a sleep time just in case the updated device state was not reflected soon enough in the API response.

9. Adding Nest and creating a Devices superclass

It was time for me to add the Nest device into the mix. For the sake of simplicity, I chose to control Nest through the Vera API, instead of the native API which was recently released. The Nest object that I created was pretty similar to the existing Light and Lock objects. Hence, I decided to create a Devices superclass, in which the Lights, Locks, and Nests would inherit from.

The Nest device was slightly more complicated, as the Vera API accounted for each Nest controller with a different id — one id for the Nest device controlling the temperature range, and another to switch between the states. I chose to expose only the id of the main Nest object controlling the temperature, and stored the other controller id as a variable within the main object class.

Upon creation of the Devices super class, I moved the update & verify state functions (functions controlling the PUT endpoints) into the class definition. This way, I had cleaner and prettier code with less repetition.

10. In comes Redis, Custom JSON Encoder, and Device Decoder

I had created endpoints to turn all the lights on and all the lights off (which I called the home and away modes). I added in the “features” to lock doors and turn the A/C down to the away mode (and a corresponding set of states for the home mode).

But manually hard coding the “Home” and “Away” modes for all the devices was not very efficient. I wanted to provide a more extensible solution for creating custom modes, where the API users would be able to store and reset the house to previous modes.

To do this, I needed to store the current states of all the devices (Lights, Locks, Nests), and be able to retrieve them when necessary. I needed some form of persistent storage… and this was when Redis came into the picture.

To store a mode, I created a new PUT method for /states/<name of slot> which stored the state of all the devices in Redis. To do that, I wrote a custom JSON Encoder which included a special “_type” variable in each representation of object to determine if it was a Light, Lock or Nest object. This was necessary as I needed to recreate the object based on its type during decoding.

Once I was able to deserialize the JSON that was stored in Redis, creating the PUT method for loading a saved state was as simple as iterating through the states, and calling the respective class function to switch the state of each device. All these was done with lots of code refactoring, of course ;)

Future plans?

So there, over a couple of weekends, I managed to create my first API to make my house a smarter home. There are definitely areas of improvements, for example:

• Using ORM for creating models instead of writing custom models
• Some kind of parallelism to set the state of the devices, locks, and Nest all at the same time instead of sequentially. This will greatly speed up the response of the API.
• Verify the state of all devices at once instead of verifying them individually as their states switch. This will also reduce the number of requests being sent to Vera.
• Ability to roll back to the original state if there is an error in switching the state of the device — This will involve saving the original state before a change is made, and then rolling it back if an error occurs.

While some of these changes can be made pretty quickly, I’m calling this done for now as I work on my next project.

What’s happening to the API?

This project doesn’t end here! Now that I've got the API ready, my housemate, Ben, is in charge of the front-end work of creating an Android app using my API.

For now, this is what the app looks like — it is very basic and lacking in design, but does exactly what we need from one place, and it’s still a work in progress.

I also heard that he had hacked up a bitcoin integration where sending money to him would result in the door opening ;)

I hope to update this with more screenshots once the app is completed! ☺

Update (2/15/2015): Omg! Someone made a pull request to my repo and cleaned up the mess! ☺

Update (4/15/2015): About 6 months ago, I bought an iPhone after being an avid Android user for the past 6 years (yes, I owned the G1!). The first thing I missed in iOS was, of course, my wonderful smarthome app. I decided to take the chance to learn some iOS programming by porting the app over to iOS, and here’s what the app looks like. The code isn’t the prettiest, but hey, it works! ;)

Acknowledgments

All this would not have been possible without the patient guidance of Ben who taught me everything I needed to know to get this far. The github repo for the API is here: https://github.com/christinang89/smarthome, although it currently only works for my house. The repo for the Android app is available at https://github.com/bencxr/kotl, and for the iOS app: https://github.com/christinang89/Unicorn

This has been fun side project. While I set out expecting to learn about creating APIs, I ended up taking away a lot more. I learnt how to sniff data packets. I learnt how to write actual code outside of a programming assignment context. I learnt about Redis/ persistent storage, and custom serializers and deserializers. I learnt many design patterns in developing a web application, some coding best practices, PEP 8 (Sorry, I know my code isn’t exactly conforming to it), and how to solve a large problem in simple, manageable pieces.

So I would say, if you’re learning how to code, learn how to make an API! ☺

My first experience creating an API was originally published in The RESTful Web on Medium, where people are continuing the conversation by highlighting and responding to this story.