The beauty of Serverless architecture is the its simplicity. You can do a lot of functionality based on a simple event got triggered from a single function. Basically, this, lets say workflow, gets triggered when for example a file was committed into a specific S3 bucket.

But, when it comes to complex workflows, its getting more challenging to maintain a healthy, successful workflow that accomplish the needful. The simple way is Lambda function Destination, which gets triggered when the main function execution completed. Of course you can trigger is when the function got completed successfully or failed (with keeping in consideration code skills to handle the function failures). But this is not an ideal scenario for complex workflow.

Lets mention several services and the sequence of evolve to achieve the best orchestration on my opinion:

SQS:

Simple Queue Service is one of the great options you need to keep in consideration. Simply it serve a waiting station lets say for the functions. You can send the event payload into the queue and then control many aspects for the execution like:

• Waiting time before consuming the message: How long should the event wait before it gets available for a function to consume it.
• FIFO: whether your workflow should maintain first in first out for any sequence execution needed.
• DLQ: if your function failed to consume the message, then send it to an investigation queue, which will hold the message for you to do your code or your sequence check. This is really good point for SQS.

But, making SQS as a trigger point for other Lambda functions can be overwhelming as you need to maintain multiple aspects for that. What can be done?

StepFunctions:

Stepfunction is a service that can work as your workload composer. This service is great to coordinate the sequence of function execution in your workload. Why this is a great service?

• State Machine: Order the execution of your functions in the order you need. What should be executed first and what should come next. Each function Payload feeds the other function to proceed with your execution.
• Parallel Execution: If one function payload suits multiple functions input, and the workflow can have multiple functions to be executed at the same time then this is a BIG plus for StepFunctions.
• A lot of compatible services: You can integrate Many services in your workflow as the compatibility is available. In fact, you can integrate DLQ from SQS into your workload as well.

This is great, but can this get better?

EventBridge:

EventBridge is an Event Bus service that has some points in common with StepFunctions but can serves wider range of workflows. Lets see why this is really good service:

• Event Routing: this is the main routing point for the service. This simply can redirect the payload into the desired destination even if the destination wasn’t Lambda function. Thanks to the set of services that are compatible with it.
• Schema Registry: This is the part that is responsible about identify, classify and trigger rules in EventBridge that can leverage the Routing service to redirect payloads into destinations. New schema version for a specific function? No problem, schema versioning is in place.
• Event Archive: This is really important feature and let me tell you why? This can make you reconcile missing events or missing execution that failed for any reason like concurrent limit exceeded or network interruption.

To summaries:

Serverless orchestration is something grow with you. You will not find the ideal scenario from day one or year one. As you can see from the sequence I mentioned the services in that not a single service is going to be an ideal solution for your workflow. Invest time in understanding these services, mirror it over your workflow, and construct what you need. I know its intimidating to leverage this much of cool services in one application but this is something you should get away from. Do not couple your workflow or at least if needed, do not do it too much.

When it comes to call and interact with other resources that need username/password, some people tends to hardcoded it in the code. Which in fact, is the worst thing you can do. For many reasons:

1- easy to gain access:

When you hardcoded your SQL server credintials in the code, you wider the range of people who can gain unauthorized access to your resources. This can be done by getting a copy of your code or even simpler, memorize it. Sounds rediculos right? Trust me I saw this before.

2- update credentials Delima:

When you have a policy states that you need to update the password quarterly, or once a year. You have to know everywhere you hardcoded this credentials, which sounds hectic. Because when you miss one place, and your system is coupled. Good luck finding it.

I can keep going giving more examples but I believe the message got received. Lets focus on AWS Lambda, one of the easiest computing unit you can code in.

Imagine with me having an application consists of 50 lambda function, which 25 functions has the value hardcoded. You changed for any reason the app DB connection. And you have 20 function out of the 50 function connect to a third party service that the policy forced you to update the password. Its one time job, but its one time job once every quarter. Who want to do this job? No one.

AWS Lambda has a verity of secret holders services that you can call to retrieve the secrets safely and within the same network. The highest security concerns you will have if you’re dumping the password into logs or you have the wrong IAM roles associated with the wrong users. I can’t think of more obvious ways that can scare you storing your passwords in one of the following services:

Environment Variables:

This is the most straight forward place they you can store your secrets in. This gets encrypted within your lambda container. How to use it? Simple, check the below captures:

Go to configuration from the main lambda function page, add your secrets:

The below example uses python code. You need to import os library and assign the value into it.

Thats it! You can see below the example of it being called and captured.

The only downside of this is that you need to update all your functions if you updated your secrets but this is a secure way to use and call your secrets. Is there a better way? Yes!

Secrets managers:

This is by far, the best way I can recommend to use when it comes to secure and use your secrets. This way can reflect into all functions that are using your secrets. It can rotate the encryption key that encrypt your secrets.

What you need to add into your Lambda IAM role is only “secretsmanager:GetSecretValue” to read the secrets.

Go to secrets manager, create a new secret and fill it with what you need.

After you save the secrets, you can validate it here:

An example of python code to call the secrets:

Simply test your function to see the secrets retrieved successfully.

This way is the best way by far. Very simple and can be called by any other AWS service. Lambda is an example of this use-case.

Bonus service to store the secrets? Yes!

Parameter Store:

This way as we can store your secrets. The thing I like about this one is that even the changes can have versions of your changes. This case insure you can always get your old secrets when it gets changed by mistake. Very simple and straight forward as well.

You need to add “ssm:GetParameters” to your Lambda IAM role.

This is an example of a parameter store:

And this is the python example to call the parameters:

And this is an example how you will get your secrets:

Summary:

You need to secure your secrets in your functions, this is the best practice. When it comes to Lambda, there are many options you can use and it’s based on your preferences, you can use your way of securing it.

In Plain English 🚀

Thank you for being a part of the In Plain English community! Before you go:

• Be sure to clap and follow the writer ️👏️️
• Follow us: X | LinkedIn | YouTube | Discord | Newsletter
• Visit our other platforms: Stackademic | CoFeed | Venture | Cubed
• More content at PlainEnglish.io

How to Use Secrets in Lambda Safely was originally published in AWS in Plain English on Medium, where people are continuing the conversation by highlighting and responding to this story.

One of the main issues that makes Lambda struggle in some scenarios is Cold Start. Which is the time that Lambda takes for the first execution.

Cold start is the first execution after a new version got published or an exection after a long inactive period of time, mainly.
This execution cause a delay in finishing the request, as it takes time for AWS to prepare the computing unit, which is relatevly fast, and to do the Init phase, which could take a long time mainly for Java becaues of the nature of code compiling.

Note: If you want to see your request from the moment it hits Lambda endpoint till the end, enable X-Ray. Its not free.

Let me add more into this point as it is the key point for this feature.
When your Lambda function set for a long period of time without executions, AWS will reclaim its space from the EC2s that are specified for Lambda. Also, if you published a new version of the same function, which it could be adding a single space into your code, will delete the current version from the EC2s and replace it with the new version. These two main point of causing the cold start.

Cool, So what SnapStart does?

SnapStart will cache the init phase internally for you. So when you call the function, it will pring the cached version to shorten the Init phase. In this case, the first execution will be way faster. As AWS shared, the cache part is free, yaay!!.

Lets dive into the Pros and Cons for this feature shall we?

Pros:

A- Speed: Lambda first execution would be much faster, especially with the heavy runtime like Java. Because the function resources initialization will be cached, Retrieving a cached image of your executable is much faster than rebuilding it every time you need to scale up or run it after inactive period.

Thats it! This feature is really to the point. It solve a big issue for many users.

I know its a new feature but lets adress the obvious points that AWS will need to deal with soon.

Cons:

A- Runtime: They support only Java for now. Which is obvious as the nature of code compliling and the number of different frameworks you can use. Other runtimes like Python is really fast to load in Lambda. Enabling this feature as well for Python makes it much powerfull and ready for more scenarios.

B- Caching: The solution cause another set of issues. Lambda is a computing unit that does what you code it to do. So, you must do a lot of testing for your code before you apply it on production workload. You much check your code for not using or relying on a set of files, saved configuration that gets pre-compiled because there is a chance that it would be using some cached values.

C- One function, multiple versions?: Because Lambda has a lot of ways to be triggered, you must update your trigger with the coorect ARN to use the version that has the SnapStart. If you don’t, it won’t use it and will use the normal function, with no SnapStart previlidge. But if this is the case, every new version, a new trigger update? hmmm no good.

I can add more points but lets focus on what is the takeaway from this feature.

It is really great feature. I’ve been using Lambda for the past 4+ years and I faced many time the cold start issue. But from my point of view, it could be better.

Like for example, add more runtimes. I know it could be a big challenge to release it with other runtimes like Python, which I can directly point to one potential issue, multiprocessing manager. Or the smartness in the trigger services to use always the SnapStart version. But these things will eventually come.

Should I use it?

If you are a Java user, I believe its a must for you to adopt this feature. It dissolve the aincient issue of Serverless. It would give you almost the same effect of running your function on an EC2, faster. But, you must do a lot of testing for your code. You built your function on a way that it works with and without Cold Start. This one might shake your code a bit.

More content at PlainEnglish.io. Sign up for our free weekly newsletter. Follow us on Twitter, LinkedIn, YouTube, and Discord.

Interested in scaling your software startup? Check out Circuit.

Should I use AWS Lambda SnapStart? was originally published in AWS in Plain English on Medium, where people are continuing the conversation by highlighting and responding to this story.

One of the most interesting architectures nowadays is event-driven. It consists of an event producer and action. So, if we imagined that I uploaded an excel sheet to S3, I want a set of actions to be performed against the records that are in this file. Pretty simple, event-driven architecture will solve the issue.

But, what makes this flow, somehow hard to maintain is that when it starts, it keeps moving. Another imagination is if you have 500k+ records in that excel file, and you had an unhandled error hits the record number 200k, what would you do? Do you reload the file and waste all computing power that processed 199,999 records? Sounds expensive and frustrating.

Luckily, there are number of ways to handle this issue. Allow me to present “ a way” to solve this issue.

It seems to be simple and repetitive? Yes it is.

Let’s assume you uploaded an excel file to S3, and you want all the rows to be cleaned from extra spaces, perform some data adjustment, and insert the records to a database.

So let me walk you through the record lifecycle, and then why it was this way:

1- File got uploaded, S3 events triggered Data Generator function.

2- Data Generator function will pass all the records into Queue 1. After visibility timeout and delivery delay reached, Function 1 will start pulling batches from the queue.

3- If a record had unhandled error, the whole batch will be sent to DLQ 1.Why is that? Because The function pulled a whole batch, not a single record. So, the batch will be still in the queue until Lambda tells the queue I’m done with it you can delete it. Otherwise, the batch will be deliver to Queue 2 and will be deleted from Queue 1 after Function 1 returns success to Queue 1.

4- The same scenario will be performed with Queue2, Function 2 and DLQ2.

Now let me tell you why this scenario is achieving the goal.

1- Event-Driven:

As you can see, no action will be performed unless it satisfies the trigger requirements. Uploading the file to S3 is the flow generator. Without it, the whole flow will be idle.

2- Lower batch to retry:

SQS FIFO (First In — First Out) can send batches that hold up to 300 messages per second or 3000 when it get sent by batches. This has some factors to it that shapes the batch size ( Data Generator passed messages interval, visibility timeout, and delivery delay). Having 300 messages to handle is more manageable than the whole file.

3- DLQ is the saviour:

Because of DLQ, the unhandled records can be isolated and dealt with separately. Some records might have some special characters, different format or even empty strings. This can toast all the processed data if it was not held in a safe place (like cache or inserted into DB). Once the records reached DLQ, you have the freedom to do whatever you want to that batch of records. Like reprocess if, for example, you exceeded the concurrent limit of Lambdas, or you had the unhandled error.

4- Price wise? Justifiable:

When we see the price of 1 million requests (0.50$) it is nothing comparing with the benefits that we’re getting from. Because this is the price we pay to isolate the records that shows us clearly the gaps and code mistakes to handle our records. Without it, you will pay way more for Lambda with the reruns that are unnecessary and A LOT MORE for CloudWatch for put logs action. Which is by the way expensive if you dont know how to use it wisely.

To conclude this short article, I highly encourage anyone who works with Serverless to invest more in this part of the architecture. Because the amount of money and time you invest to handle these small, yet expensive errors, is totally worth it. I will bring part 2 to show you how you can make it more dynamic with EventBridge. Until then, be safe.

More content at PlainEnglish.io. Sign up for our free weekly newsletter. Follow us on Twitter, LinkedIn, YouTube, and Discord. Interested in Growth Hacking? Check out Circuit.

How to Handle Asynchronous Requests in AWS was originally published in AWS in Plain English on Medium, where people are continuing the conversation by highlighting and responding to this story.

AWS is amazing, the services and the interconnection parts make it fun and really enjoyable to work with, even for fun projects like this.

Do you want to try it? Click here

So, I was bored and decided: I want to launch EC2 with voice! Pretty simple. The diagram will show you how:

The diagram shows the whole story, but lets describe it a bit.

1- The record:

The first step is to create a unified statement that you want your transcription job to turn into text, which will be processed by the function.

For this example, I used a website to record my voice and to download it as mp3.

2- Upload the file:

Once I got the record, I uploaded it into S3 and let it work. I configured my bucket to send S3 payload into Lambda that I created to deal with what gets passed from this bucket action.

Then, Lambda and Transcribe time come.

3- Lambda time:

Once the payload triggers the function, I get what I need from the payload to pass it into Transcribe. These are straight forward steps.

I let it to identify the language because I didn’t know the recorded voice will be with which accent. If I recorded it with my accent, I believe it will get a hard time understanding the Arabic accent of my English.

transcribeJob = transcribe.start_transcription_job(

                TranscriptionJobName=transcribeJobName,

                Media={
                     'MediaFileUri': 's3://'+bucketName+'/'+fileName
                       },
                IdentifyLanguage= True
)

Then, waiting for Transcribe to finish its job.

4- Transcribe:

Transcribe is a fast service, despite that my record is short, the response you get is relatively fast.

So, I get the job status and look for `TranscriptionJobStatus`. Once I get `Completed`, I get the link that holds the text for my job.

This is an example from the console:

I do some basic verification over the words as I set the function to get some keywords like Create and Option to identify the action I want to do.

The next step is to check the resource that I want to make. I hardcoded it as I did the minimal for this example. But, as always, the sky is the limit.

5- Creation time:

Once all the previous steps are done, I loop over the pre-config list that I made and get the parameters to pass it into the command. In my case here is creating an EC2 instance.

And here you go, an instance got created because of a voice record.

To conclude:

There are many things you can do with event-driven architecture. This is a small, fun, waste of time for some, but great evidence that this kind of architecture is powerful. I did lambda execution via SMS before and it was fun. Check it out if you’re interested in this kind of unusual actions with AWS Lambda.

More content at PlainEnglish.io. Sign up for our free weekly newsletter. Follow us on Twitter and LinkedIn. Check out our Community Discord and join our Talent Collective.

How Did I Create EC2 by Voice? was originally published in AWS in Plain English on Medium, where people are continuing the conversation by highlighting and responding to this story.

ETL stands for: “Extract, Transform, Load”, which is the process of dealing with a series of data in a computing unit.

Since AWS Lambda is a computing unit, cheap to use, and versatile with what you can achieve or combine with, makes it an appealing option for this kind of jobs. But does it?

1- What ETL Jobs need to finish successfully?

Let’s assume that you have an excel sheet with a huge amount of rows along with a respectful number of columns, and you need to clean, transform and store these rows in a specific order or shape. You need to have certain components to achieve this task.

AWS Lambda provides these sets by being able to connect with storage services like S3, Glacier. Also, you can use EventBridge to create a series of actions that fits with the custom schemas you’ve configured. DynamoDB gives the ability to use NoSQL option that provides a pretty fast DB connectivity. SQS for queueing the data into the function, and catch the unprocessed data for later adjustment in the DLQ.

So, we can assure that the underlying architecture and connectivity is there for Lambda to do the job.

2- Only Lambda can do the job?

Of course NO. AWS has another service that called Glue, which is a Data Integration service that built to do these type of operation. It has its own set of features and options. But it cannot be compared to Lambda when it comes to the connectivity. Lambda has more options. Yes Lambda is a computing unit and Glue declared as Data Integration service. But, what we’re doing here is to compare the use case itself, not the limitations.

What can be done in Lambda, can be done in any EC2 family. You just need to find the right family to do the job. But, keep in mind that EC2 is a customer managed service, you have to take care of.

One small comment here, if your ETL is consuming a lot of CPU, do not use T family.

Another point I want to clarify that with what Lambda can utalize, it can recover the job if an issue comes to the code like unexpected data type like a difference in datetime format. Lambda can redirect these rows into DLQ which can work as a net that catch the uncommon data rows. This connectivity isn’t presented clearly in other services that might made you rerun the whole file after fixing the code manually. Trust me I’ve been there.

3- Advantages vs Disadvantages:

Let’s start with with the disadvantages:

• Time: you cannot run the job normally for more than 15 minutes. It needs more than that to split the file, run many functions and maintain the passed data integrity.
• Power: it is very limited to what you can allocate, it is hard to know the right amount of memory to allocate which comes to CPU allocation.
• Misconfiguration: since its a computing unit, your code might be the misconfiguration. Using the wrong function in your code could lead to many issues which affects the allocation of memory and create bottleneck.

Let’s lift the spirit a bit:

• Less configuration: Since its a managed service, you don’t need to think about the availability of service, or if any other process will use the computing power that you need for your job. It’s all taken care of.
• Ready runtimes: If you need to use python, just select it and all needed tools, services are ready to be used. As simple as that.
• sidekick services: You can use a lot of services to help you achieve the goal easily. Thanks to the SDK.

So, the question is does it work for ETL?

The answer is: it depends.

It depends on what are you doing, what are the expected outcome and how are you utalizing it. It can be very costly if you’re using a lot of surrounding services. Or your code it taking a long time to run. All these factors can influence your decision.

There are no perfect services for ETL. There are services can do the job. Do you know what kind of job is needed to accomplished? Can you picture the process? Then you can answer this question.

More content at PlainEnglish.io. Sign up for our free weekly newsletter. Follow us on Twitter and LinkedIn. Check out our Community Discord and join our Talent Collective.

Is AWS Lambda Good for ETL Jobs? was originally published in AWS in Plain English on Medium, where people are continuing the conversation by highlighting and responding to this story.

For a while, I struggled when it comes to making layers for my functions. I used to download them locally, zip them, upload them into S3 when their sizes are big, and then create a version. This process takes a long time and the changes you make to a defective layer are high when you’re using Windows or Mac because of some layers that compile binaries when you download them.

I wrote an article back in 2019 on how to do that with the help of Docker.

How to Compile Resources for AWS Lambda?

Since then, I still get reads for it. Which triggers the need to create a new simple and extremely fast way to use this amazing feature, Lambda Layers.

I will walk you through what it does and how you can have it in your account. Please note that this process is only (currently) for Python3.8.

What is does?

This script consists of 3 main steps: Create a new layer, update the existing one, and read what’s inside your latest layer version.

Create a new layer:

Because of the struggle I mentioned at first, this process is time-consuming when it comes to the manual, traditional way. So, with providing some key information, the script will create the directory structure, install the libraries with PIP, calculate the direct size to prevent exceeding layer limit, zip it, upload it into newly created S3 bucket (or existing if you have one), and finally, publishing the new layer.

these steps are the minimal that you can do to create a new layer. Please note that some values are hardcoded, it can be easily made dynamic but it's out of the scope (currently).

Update existing layer:

Because managing an existing layer could be hard, as you will need to do many steps to maintain the existing libraries and add new ones. This script will get the referenced zip file of the latest version of your layer, download it, add to it, upload it into S3 again, and publish a new layer version.

Read Layer content:

Because of the problem that log4j made a couple of weeks ago, and what threat can existing resources made if they were got affected, you will need to monitor your resources and update them accordingly. This action will get the latest layer version zip file, extract it, use PIP to check what’s inside and which version you have. This could also help maintain supported libraries versions.

Enough talking, let's dive into it:

Preparation and execution:

We will go through the steps in order:

S3 steps(if you don’t have one already):

1- Login into your AWS account and go to S3.

2- Create a new S3 bucket, keep it in the same region you work in.

3- Set it up as you wish, no red lines are here.

Lambda steps:

1- Go to the Lambda console and create a new function.

2- Open the function -> Click on Configuration -> Click on Permissions -> click on Role Name.

3- Click on Policy Name -> Edit policy.

4- Paste this policy (edit resource as you wish) -> save it.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "lambda:ListFunctions",
                "lambda:ListLayerVersions",
                "lambda:ListLayers"
            ],
            "Resource": "*"
        },
        {
            "Sid": "VisualEditor1",
            "Effect": "Allow",
            "Action": [
                "lambda:GetLayerVersion",
                "lambda:DeleteLayerVersion",
                "lambda:AddLayerVersionPermission"
            ],
            "*"
        },
        {
            "Sid": "VisualEditor2",
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:GetObjectAcl",
                "s3:GetObject",
                "logs:CreateLogStream",
                "lambda:PublishLayerVersion",
                "s3:ListBucket",
                "logs:PutLogEvents"
            ],
            "Resource": [
                "*"
            ]
        }
    ]
}

5- Go to GitHub and get the code -> to copy the whole code in code.py and paste it into your function.

GitHub - imohd23/lambda_layer_maker_python3.8: This is a script to make create, update and ready AWS Lambda layers you have in your account.

6- Click on Test and paste this json (all fields are required)

{
  "layer_name": "LAYER NAME",
  "s3_bucket": "YOUR S3 BUCKET",
  "libraries": ["LIBRARIES"],
  "action": "create_new"
}

NOTE: in the action key, there are 3 valid values: create_new, update, read_only

7- Test the function is what triggers it, give it a try.

These are proofs of this script execution:

Feel free to adjust the code as you wish, and let me know if you have any issues.

Conclusion:

This is a way of doing this procedure, I’m pretty sure you have your own way too. let me know, does it worth it? Do you know how to code in Node.js or Java and wish to see this script take another turn and provide another language's support? I’ll be thrilled if that happened.

Stay Safe.

Thank you for reading.

More content at plainenglish.io. Sign up for our free weekly newsletter. Get exclusive access to writing opportunities and advice in our community Discord.

Create Lambda Layers in AWS Lambda was originally published in AWS in Plain English on Medium, where people are continuing the conversation by highlighting and responding to this story.

Security is no joke, one of the key things you have to visit whenever you’re architecting a new serverless application.

Serverless application is basically an idea, like any other application, that you have and it has a logic. You need a computing power to execute your logic, an object storage in case you need it, a place to store your information and a way to tell your logic: go go go!

Pretty simple!

The beauty of Serverless is that we don’t care where and why our logic got stored the way it does, although Information Security teams would get really mad if they read this statement. Anyway, what really matters for us as a cloud developers, as I like to name ourself, is that our logic is getting executed, customers are happy, win-win situation!

But, I wish it was that way. We all need to care about our own applications security. Some people would say: Security! Hack! We got exposed! Well, sometimes these things happen. But, you as a cloud developer, can help preventing this from happening. Let’s dive into it together shall we?

Standard Services

Most basic Serverless applications would need the following services:

1. Execution endpoint -> API GW
2. Object storage -> S3
3. Computing unit -> Lambda
4. Records storage -> DynamoDB

Of course you can have more than this, but let’s keep focusing on the main parts that shape the basic Serverless application.

All these services are managed by AWS, in case of batching, updating or deprecating runtime, scaling services based on our usage. All the major tasks are handled by them. Which is the soul of this architecture, focusing on developing and leaving operation tasks for them. Which I’m really fine with. But they can’t do the job without us, the cloud developers.

We have to care about our applications. We must secure it, which in fact, is pretty simple.

Securing your Serverless application starts at the beginning, best practice.

Wait, are you telling me that if I followed the best practice per service, that would secure my application? Yes!

Let’s take a closer look on some of these steps per service.

API Gateway (API GW)

Use WAF

AWS provides a service called WAF, which is a Web Application Firewall. Self explanatory name that tells you why to use this service. It provides a protection against harmful usage of your resources that might affect the availability of your application and consume its resources, which will lead to interrupting the usability of the application .

Use WAF to monitor your API GW executions and act based on specific metrics. Using pre-defined roles would save you time planning and testing techniques to prevent common online threats.

Configure Endpoint Throttling

You know your application and your customers, if an endpoint is getting unusual executions, this means that there is something wrong. Setting up throttling threshold would stop the exhaustion of your resources, especially if this endpoint is executing Lambda, which will visit in a minute.

Validate Your API GW Execution

One of the major risks in Serverless applications is broken-authentication. If your endpoints are being called and consumed by unauthorized actor, this means you’re in a big trouble.

Use services like AWS Cognito to authenticate the executer. In this case, you would have a better understanding about your customers and to insure that no one will execute a function that he’s not supposed to.

S3

Block Public Access (in some cases)

If you’re hosting sensitive data like identity cards, passports, secret reports, you have to block public access to this bucket. This will prevent any attempts to expose your files.

But, this recommendation won’t work if you’re hosting publicly available data in your bucket as it needs public access. Yes, there are some other options to share these files publicly while it’s publicly blocked. But, why the hassle? If you reached this point, you need a help on your S3 strategy.

Encrypt At-Rest And On-Transit Files

The main 3 operations against S3 are: putting object, store it and eventually retreive it. So, to do these 3 operations securly, consider the following:

To upload securely into S3, use presigned url for that. The reason for it is that you, as a cloud developer, allow PUT action into a specific bucket for a specific file. In this way, you give the API an authority just to do a single action on a single file. Less privilege less concerns.

To store the file, you have to enable server-side encryption. This recommendation comes from AWS themself as they advice this move to enhance the security level and privacy. Also, it comes with no additional cost. Cool right?

To share the file, consider the presigned url, the same concept as upload. When you generate a presigned link for sharing a file, it gives you a url that consist of object path and an access token that is valid to retrieve only that specific file for a period of time. Once it gets expired, it is unuseable.

IAM Role Access

Creating an IAM role is really one of the best “best practice” actions that you as a cloud developer needs to enhance. For S3 case, creating one with specific actions would limit the level of S3 bucket/content exposure.

Create a role that has the specific action on the specific bucket. Why? Because this will make sure, even in case of compromised access, that this access won’t cause a great harm on your data.

Bucket Policy

Bucket policy is a list of rules that guards the bucket from misused intentionally or unintentionally. I had a hard time understanding this concept and some examples need articles to describe but lets summaries it in the next example:

You have a bucket that can accept files generated only from any resources that is deployed in “Core-Production-VPC”. Usually we guard our bucket with an IAM role, but in this case, it’s not the suitable thing as anyone who can have an IAM role allowing actions against this bucket will be able to perform actions against it. So, we create an S3 bucket policy that can accept PutObject only from “Core-Production-VPC”.

Who will win? IAM role or Bucket Policy in this case? Well, it’s a combination of both. If your IAM role has an access to this bucket, the call to this bucket will pass through. But, if you don’t fulfilled policy requirements, then sorry, you can’t PutObject in it.

Really cool way to make sure you don’t have misconfiguration or misuse from IAM roles.

Lambda

One IAM Role Per Function

Lambda has an access to many services and when you use SDK, oh man, you have them all! So, should you be happy? NO, because the risks are countless.

So, let’s focus on this as its really important point. Less privilege less concerns. AWS advice that you limit your function with a unique, sophisticated IAM role privileges. In this case, you can be sure that this function won’t cause unexpected harm as its boundary is well defined and known. One of the new security risks is Event Injection. Did you test your function well? Think twice.

Temporary Credentials For AWS Access

As we said in the previous point, Lambda can access almost all your AWS resources. Tell this information to your Information Security team and run away! So, How to set its access boundaries ? In API GW section, we talked about authorization, and yes it applies here. You have an application that one of your customers stores data into DynamoDB. You generate a short living token to be used from this customer with the specific target. Once the time comes, the access is revoked. Simple and affective.

Limit Execution Duration

When it comes to execution duration, it comes to what you actually pay AWS. As Lambda is pay-as-you-use, you need to know what you’re paying for.

One of the concerns is that if your functions are under DDoS attack (which is not good), your function will be running all the time. This has a financial and usability impacts. But, to reduce the affect from these kind of attacks is to limit execution duration to what your tests show. Test the function, know how long usually needs, and set it as a hard limit for your team. Function will timeout and will free up the resources. But, this would make a bigger attack rates? Not necessarily as you might face a small amount of load, which can reduce the amount of affected customer. How?

Ok, if you’re under attack, your resources are bing executed in a high rate calls, which could make them busy. To avoid that, to a certain rate, limit the execution. This will result to quickly release them and free up the place for “some” customers calls. It seems not to be a great option but if the attacker reached the functionality level, you don’t have that much of options. Do you?

Coding Best Practice

Your coding practices can have a great impact on your security. How? Let’s find out.

Let’s assume you have a wrong IAM role attached to your function, and you’re using a 3rd party library that has vulnerabilities, guess what can happen? Always choose a trusted 3rd party libraries.

If you have a gigantic function that had an enormous payload. The passed payload has an event injection script and you didn’t validate it. Man you’re doomed. Always validate your payload.

I can keep writing about this section forever. But I believe the message received. Best practice can save you from threats.

DynamoDB

Encrypt At-Rest Data

So, when you have an important data that you need to store in your table, like sensitive information about the clients, or your customer business secrets, you defiantly need to encrypt them in your table. The reason is to protect these data from unauthorized access via your API or even from your developers! Yes, sometimes the risk can come from inside.

How to store your keys? Well, you have many options, like System Manager, Secret Manager or in your lambda environment variables.

IAM Roles to Access DynamoDB

This point is not new, you have to lock the access into your tables with IAM role. In this case, you, as we described earlier, can insure no function will do anything that is not supposed to do, which will decrease the surface of potential harm from your functions that could be a victim to any Event Injection attack.

VPC endpoint to access DynamoDB

If you have functions that get deployed within a specific VPC and only these are being able to access a specific table, you can setup a policy that allows the access from only the wanted VPC. Cool! This also will decrease the attack surface as if one of your functions that belongs to another source got compromised, you’re good to some extend as not everything in DynamoDB will be exposed to the attacker.

Summary

So, to summarize, following these steps “will” enhance your Serverless application security as you cover most of the main points. As you can see, these are really not hard steps. It has a lot of documentation behind it and it will enhance the team standards. It is a win-win situation.

Stay Safe!

More content at plainenglish.io

How to Secure an AWS Serverless Application was originally published in AWS in Plain English on Medium, where people are continuing the conversation by highlighting and responding to this story.

Processing data could be an intensive task, especially on the computing units as the read and write operations takes a lot of resources. Luckily, if you have the right tools, you can achieve it easily. But, is it worthy? Let’s find out.

In this article I will share my experience with you on how did I achieved that. It is really simple and complicated at the same time. Why? Because of how Lambda works and what you have to think of when you “Code” because that really makes a difference.

Why did I thought about it?

Few years ago, my manager told me to think about a processing architecture to process BIG volumes of records but not that heavy operations. Like 800k rows of data, with 16 columns, that the amount of work needs to be done over each row isn’t complicated. So, Event-Driven Architecture!!

I went through a lot of issues on how to deal with the limited resources in Lambda and how to deal with my records dropping because of timing out and OS errors. S3 was another story also to learn how to tune it for my use-case. A dear friend who works as senior consultant in AWS Bahrain helped me getting some tools in place to achieve this promising idea. It was one of the best experiences I ever got dealing with AWS resources.

Enough talking let’s get some diagram in place.

Solution Diagram

Diagram looks scary? Trust me, it isn’t.

Let me break it down for you in some steps:

1. Initiate the process

Because I’m adopting Serverless architecture, it means Event-Driven events that if something happened, things will act based on it and the results will trigger another actions, till the end of the process.

So, in our case is S3 Put request. When we upload the file into S3, it will put the file inside a bucket and when the file fully uploaded into it, Lambda will get triggered with S3 payload. Our first step just got finished. Whats next?

2. Clean data

Because we got a csv file, some columns and rows could contain spaces, some special characters that MIGHT break your code. So, clean it up.

Cleaning these records will prepare it to be ready for insertion. But, since we have a lot of records and the function might fail, how to track what was added and what left over?

3. Add clean data into a queue

We will add the cleaned records into a queue. The reason is to track what has been added and what did not got added. Basically, SQS will act as an organizer. It will send small batches into Lambda, Lambda will add it into DynamoDB, then it will return success message to SQS to remove it from the queue.

What will happen in case of failed record is SQS will retry 3 times, as per my configuration, to try insert operation. If these 3 tries failed, then it will move it into Dead Letter Queue (DLQ) which is another SQS queue that have the failed records. Then, you can debug why these records never make it into DynamoDB and can be processed again or even reject it.

4. DynamoDB

Because we are trying to process massive chunks of data, we need some sort of Database that can handle the extreme load or records. DynamoDB solves this issue. There were a lot of experiments on how to handle the number of records and how to behave with the limited read/write throughput as write can handle 1kb data per unit. So, DynamoDB on-demand solve the issue.

As per AWS documentation, using on-demand DynamoDB throughput is the option when you cannot predict your workload. Because it will prepare the max throughput just in case its needed.

We moved the records from csv to DynamoDB, then what?

5. Stream records to SQS

DynamoDB is really good event executer for Lambda. When you enable Stream, you can specify a Lambda function that react to your passed payloads from it. The good thing is you need to act based on type of record. We are dealing now with the newly added records. So, when we verify the tag, we add the record into another SQS.

The reason for this queue is the consistency of records delivery. You are adding the record once, capture it and add it into the queue so you can deal with it. If not, then you have to scan the table to get unprocessed records and process them. Why the hassle? Let the queue deal with it.

6. Process data

We reached the latest stages of record lifecycle in this architecture. When it reaches the Process Queue, it pass the records in batches, process them, and then pass them to another queue. As I clarified earlier, for consistency purposes.

7. Update the processed record

Finally, the record will be grabbed from the Finished queue and passed into the Lambda function that will update the record with the processed information. If records failed to be passed, DLQ will gather them for your further debugging and actions.

Challenges

These points seems to be a straight-forward scenario, BUT, its not. Let me walk you through some problems and how it got solved.

1. Lambda Lambda Lambda

Lambda is the key player here, we have limited time to execute the logic in your code. How can you insure the records were red from the file, cleaned, and added into the queue? Its hard but what you will need is speed. I code in Python and I used Multiprocessing library to speed things up. I used Multiprocessing Process function to use every single possible processing unit in Lambda. This action made my process to clean (in some tests) 558k transaction in 1:30 min! It was really fast. Again, it is not straight-forward scenario. Lambda can handle around 500 process when you allocate the max memory. Any other process will raise “OS Error 38: too many files open”. Why did I faced this issue? Because I joined all running processes and it was not closing finished processes. So, I run half the batch and loop over the running processes. If it finished, I force it to close. Problem solved.

2. Keep an eye on CloudWatch

I made a big mistake by passing event variable into CloudWatch even when I was running the big batches. This resulted to write 6.6TB of data because of my many tests. The price of CloudWatch put log action is expensive so use it wisely.

3. DynamoDB on-demand is the keyword

I started my configuration with making my DynamoDB provisioned, which is something for predicted workload. I made it 5 read and write throughput and that was an issue. I faced the issue that out of 558k records, only 1k was inserted into my table. I raised it into 100 throughput and still at least 60% out of the file get lost and not added! Then I reread the documentation and noticed my issue, DynamoDB on-demand is the solution for unpredicted load. I added all the 558k+ records within 5 min! Pretty FAST!

4. SQS can be tricky

SQS is a great service and has a lot options and opportunities. But, you need to know what is the size of the batch you’re passing every time and what is the predicted time for your batch to finish. The reason for that is when you tell SQS to wait x seconds before making this batch available again, it might get processed multiple times. Know your code and your data, test test test and then configure it for heavy workload.

These points were my top concerns, S3 was interesting but not that complicated as I expected. But the main question, does it worth it?

Conclusion

Everything in this life depends on conditions, if you don’t want to manage instances or you want it with the minimal effort then yes, this scenario is valid for you. Keep in mind that debugging these use-cases can be tense because of how connected and how one mistake in one step can affect the coming steps.

Stay Safe.

I Processed 558k Transactions on AWS Lambda in 5 Minutes was originally published in AWS in Plain English on Medium, where people are continuing the conversation by highlighting and responding to this story.

For many users, preparing an environment in AWS to host their website/system was tough, especially if you were new to Cloud.

Also, even after doing the preparation, the creation of resources might take a while too! But, what if you can do the following within the 20-minute window? Check out the list:

• Create a VPC and all essentials for public and private subnets.
• Create an EFS and prepare an access point to this storage option.
• Create a launch template to replicate your configuration into multiple EC2s in different subnets.
• Prepare your website as Docker image to have consistent replica of your main concern (your website and its DB).

It feels wired right? Actually no, because this is the beauty of IAC and Terraform. Why don’t we go deeper? Prepare your swimming suit.

NOTE: This is a POC script and got tested, yes it can be better but for the article purpose, I guess this is a good starting point.

What you will get?

Check the image below, do not get tricked yet, wait for the clarafications:

After you run the code “ as it is”, you will get a load balancer placed in one of the public subnets, one EC2 machine that got deployed by Auto Scaling Launch Template and EFS.

These things are enough to run your website, at least for this use case that I am talking about.

But what actually happened when I run the script is the following:

• For VPC, the script will make a VPC, subnets (public and private), routing tables and internet gateway.
• For Auto Scaling, the script will make a launching template with what is needed to replicate the configuration into multiple instances as it required and will make sure that the minimum required instances are always meet the configuration.
• For Load Balancer, the script will prepare an endpoint associated with health check metrics.
• For EC2, the script will launch an instance from the configuration script that got attached to the Auto Scaling launching template, this will install and setup Docker inside the machine and will pull the project from GitHub.
• For EFS, the script will create all needed resources like security group, access point, and access point attachment for this storage.

Is that too much? I guess not, some other systems need a lot more. But, let us agree on one thing, doing all these things in 20 min manually is exhausting.

The steps of deployment:

It is really easy and needs few clicks, let us start, shall we?

Before we start, clone this link.

1- Do you have Terraform installed?

First thing first, check if you have Terraform installed in your machine.

2- Do you have your AWS credentials?

Because we are going to deploy this instruction to AWS, make sure one of these two options are available:

• AWS CLI is configured, So you should only know which profile will use in case of multiple configurations are placed.
• AWS access and secret keys, because of course, we need credentials to access.

NOTE: Check provider.tf once you decide which connection to use.

3- Some necessary commands.

Open a terminal window and navigate into the project.

Because we are going to make a new instance and we might need to access the instance, so run these commands.

ssh-keygen -t rsa -b 4096 -C "your_email@example.com" -f $PWD/id_rsa

NOTE: Do not change the key name unless you go to vars.ft and change it there also.

Then, run this command to get all needed modules:

terraform init

4- Trigger the script.

There is only one step left, maybe two if you want to check what will happen.

To check what are the resources that will be created, run this command:

terraform plan

To bring all these planned resources to the life, run this command:

terraform apply

After all resources get aligned in the correct order, a message will ask you to approve these changes, type yes.

And now you can relax till the whole process is finished.

NOTE: After Terraform script is done, please wait 5–10 min before checking the website, the reason is that the infrastructure is done but the instance script is in progress. EFS mounting needs some time to be done.

To clean everything, run this command:

terraform destroy

Important points:

1- Why EFS in this simple use case? Because I am using Docker to host the website and DB all together, and you have an auto-scaling launch template, EFS should insure the consistency of your DB because the DB has a volume pointed into EFS. Any new instance will use EFS as volume.

2- At the end of the script execution, Terraform will return an ELB link and this will not work directly. Why? Because WordPress's first initial page is going to be /admin to finish all needed configuration. So, go to aws console -> EC2 -> your created instance and visit its public IP to finish the setup. Then all going to be good.

Summary:

I don’t know if this process is going to be helpful for anyone, but since I know how to make it, why not share it?

Creating a full basic environment in AWS can be difficult, especially for newcomers to this platform. This script will give them few important points to think about and show why using Docker is a really good option, why using Terraform is not hard, and finally, why IAC is something they need to work on.

More content at plainenglish.io

How to Deploy a Full Environment for WordPress via Docker using Terraform was originally published in AWS in Plain English on Medium, where people are continuing the conversation by highlighting and responding to this story.