Sitemap

Learning AWS Lambda

5 min readFeb 20, 2017

--

Today I wanted to write a quick piece about my recent experiences learning and deploying my first app to AWS Lambda.

Firstly what is Lambda? There is plenty of reading material available on the topic, but the best description of Lambda is that it is a Function as a Service (FaaS). Essentially you can deploy a function to AWS (in my case written in Node.js) and run this function on Amazon’s infrastructure.

On its own this might not be particularly useful, you could offload some of the processing that happens on your machine/webserver onto AWS but you’re then introducing latency into the mix so depending on your use case that may not be that wonderful. But what does make Lambda really exciting is an service called API Gateway. What this allows you to do is create an API endpoint that can then connect to the function you’ve deployed on Lambda. If it wasn’t obvious by now this means you no longer need a webserver!

Well not exactly… you still have a web server, but the difference here is that you no longer write the code for it, AWS provides this for you. Instead you need to do some configuration to specify what endpoints you need and what data from the request (query strings, params, body etc…) your function needs to run. You can configure this directly in AWS yourself but as your spider sense is probably telling you right now, the AWS UI is not the friendliest of beasts so the configuration is complex and spread across a number of difference services you would need to setup to make it all hang together.

To address this a number of frameworks exist that handle the heavy lifting of configuration for you. I have been using the Serverless framework and recommend watching this setup video on how to configure the CLI with your AWS credentials. Basically Serverless replaces the manual AWS configuration you’d otherwise have to do with a .yml file within your project that instructs Serverless how to configure your lambda function. Then, anytime you make changes to the function, it will then update your configuration accordingly on your next deploy.

So what next? I recommend watching the next two videos in the series which will show you how to generate a serverless project and a lambda function, then the final video will show you how to connect this to an HTTP Endpoint. From here you should have enough knowledge to think about deploying your own project to AWS. In my case I wanted to build a Kickstarter Comment scraper as my first comment and was able to fairly easily deploy this to AWS.

Once you have Serverless setup it’s incredibly easy to deploy as well as run / test your lambda functions locally as well as on AWS. When you run the deploy command the folder where your .yml file resides is zipped and sent to AWS for deployment. This means any modules you’ve installed are included in the zip file so deployment can be slow but it’s entirely self contained and Lambda therefore doesn’t need to install anything itself (no npm install, shrinkwrap issues etc…). You can configure folder’s to ignore in the .yml file if they are not needed for deployment as well i.e. /src /.git etc…

Get Simon Taylor’s stories in your inbox

Join Medium for free to get updates from this writer.

There are a few gotchas I experienced along the way though so I’d like to point these out.

  • Currently Lambda supports node version 4.3, for more recent versions of node you’ll need to use Babel. I was initially getting a weird error which took me a while to realise the issue was with using new language features that were not available in node 4 (e.g. object destructuring and template strings).
  • You can test (invoke) your lambda functions locally as well as on AWS without using an endpoint. Getting your function working on Lambda should start with local testing, remote testing and finally via curl in that order. One issue I was experiencing was when you deploy your function to lambda and configure API gateway to pass query strings / params / body etc… this is provided to your handler function via the events object. However in the CLI you can only pass a string into your handler function’s event argument which prevents you from testing locally. But eventually I discovered you can create a JSON file with the contents you want to use for the event argument and then specify a path to the JSON file which then gives you an event object.
  • When I was having issues with my Lambda function I wanted to check the AWS UI to at least see if it was configuring my function and endpoints correctly. But when viewing Lambda functions on AWS I could not see my function. For anyone familiar with AWS this may be obvious but the reason this was the case is by default Serverless deploys to the us-east-1 region and in AWS console I was viewing a different region. This can be changed in the serverless.yml but in my case I was just confused why it wasn’t visible.
  • Finally, stating the obvious here but the serverless.yml is a dumb configuration file. Meaning if your syntax is wrong, well shit just won’t work. In my case I had omitted integration: lambda from my http events which meant my endpoint was just not working. After a while I finally stumbled across this in the docs and fixed it.

My particular use case as mentioned prior is a kickstarter comment scraper and it’s a great use case for Lambda, but I’m still thinking there will be future challenges.

  • I’ll probably need to connect it to a database to cache scraped comments which will probably be tricky as I’ll have to learn DynamoDB or at least run some other database somewhere that is accessible to my lambda function.
  • I also know that Lambda has a 5 minute timeout in built, meaning if scraping the comments for a single kickstarter project takes longer than that, it will probably just kill the process. This is problematic because kickstarter comment fetching is happening in sequence due to one page of comments providing me the url for the next.

But ultimately I am massively impressed by just how easy it is to get your code up and running on AWS with little to no devops knowledge and for some use cases it is the perfect tool for deployment. I’m more than happy to spend the time learning some new tooling if it means I can deploy my own code with ease and that’s not even considering the other benefits Lambda provides such as scale and cost that are covered in much more depth elsewhere. Anyway give it a go and tell me what you think!

My code for my kickstarter comments scraper with serverless config is available here. What’s awesome is aside from the code I’d already written to scrape kickstarter, the Lambda code / Serverless config is literally just two tiny files with very little in them.

--

--