AWS Lambda – Build and Deployment Strategies

AWS Lambda – Build and Deployment Strategies

Serverless – A short primer

Serverless is essentially being able to run your application/code ‘somewhere’ and more importantly not be concerned about ‘where’ it runs. The ‘where’ here is not just in reference to location but also the hardware and operating system. Additionally, serverless is meant to scale, infinitely, at least in theory.

Serverless has its beginnings as early as 2008, when Google released Google App Engine. Today, Serverless computing is available from a number of providers, supporting various technologies and with a variety of price points

AWS Lambda

Amazon launched “AWS Lambda” service in November 2015. AWS Lambda provides an event-driven serverless computing platform. With Lambda, you can write programs in your preferred programming language and deploy and run it without having to worry about underlying system/location/hardware constraints.

We regularly develop and run a number of Lambda functions both for FourCo and for our customers. Although, writing a simple Lamba function is mostly quick and painless, it can quickly get a bit more complex if the function needs to be modified regularly. Setting up a CI/CD pipeline can not only rescue us from these scenarios, but also make development and deployment a lot more seemless especially with complex functions and scenarios.

More than one way to do it

There are various ways you can deploy an AWS Lambda function. We will discuss the most popular methods of deployment:

  • Cloudformation (CFM) templates
  • AWS CDK
  • AWS SAM (Serverless Application Model)
  • Serverless framework

The first three methods are all AWS way of doing it.

Using CFM Templates

You may already be familiar with Cloudformation templates. These are yaml or json based templates to declare resources. Using CFM, you can declare Lambda and associated resources (IAM Roles, policies, cloudwatch events to trigger the Lambda, etc.)

A sample cloudformation template declaration (in yaml) should look like this:

AWSTemplateFormatVersion: 2010-09-09
Description: A Sample Lambda Stack (FourCo IT Services)

Resources:

  RoleAttachedToLambda:          
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              Service:
              - lambda.amazonaws.com
            Action:
              - 'sts:AssumeRole'
      Path: /    
      Policies: 
        - PolicyName: LambdaIAMPolicy
          PolicyDocument:
            Version: 2012-10-17
            Statement:
              - Effect: Allow
                Action:
                  - 'sts:AssumeRole'
                Resource: 'arn:aws:iam::555555555555:role/SendEmailRole'
              - Effect: Allow
                Action:
                  - 's3:GetObject'
                  - 's3:PutObject'
                  - 'sqs:ReceiveMessage'
                Resource: '*'
              - Effect: Allow
                Action:
                  - 'logs:CreateLogGroup'
                Resource: '*'
              - Effect: Allow
                Action:
                  - 'logs:CreateLogStream'
                  - 'logs:PutLogEvents'
                Resource: '*'        
      RoleName: LambdaIAMRole    

  OurLambdaFunctionResource:         
    Type: AWS::Lambda::Function       
    Properties: 
      Code:
        S3Bucket: s3-bucket-where-code-is-uploaded
        S3Key: lambda-function-code-as-zipfile.zip
      Description: Lambda function to process to notifications
      FunctionName: HelloCloud
      Handler: lambda_function.lambda_handler
      Role: !GetAtt RoleAttachedToLambda.Arn
      Runtime: python3.8
      Environment:
        Variables:
          EMAIL_RECIPIENT: operators@fourco.nl
          WEEKLY_EVENT_SQS_QUEUE: WeeklyQueue
      Timeout: 15
      
  LambdaFunctionScheduleRule: 
    Type: AWS::Events::Rule
    Properties: 
      Description: Invoke our Lambda function every day at 6AM
      ScheduleExpression: cron(0 6 * * ? *)
      State: ENABLED
      Targets: 
        - 
          Arn: !GetAtt OurLambdaFunctionResource.Arn
          Id: LambdaFunctionScheduler
          
  PermissionForEventsToInvokeLambda: 
    Type: AWS::Lambda::Permission
    Properties: 
      FunctionName: 
        Ref: OurLambdaFunctionResource
      Action: lambda:InvokeFunction
      Principal: events.amazonaws.com
      SourceArn: !GetAtt LambdaFunctionScheduleRule.Arn

Pre-requisites

  1. AWS CLI (optional – actions can also be performed manually through the AWS Console)

Steps to deployment

  1. Package your code
  2. Upload it to S3 bucket
  3. Deploy the changes to the stack through aws cli
aws s3 cp <packaged_lambda_zip> s3://<bucket_containing_lambda_zip>

aws cloudformation deploy --template <json_or_yaml_cfm_template> --stack-name <cfm_stack_name> --parameter-overrides S3BucketName=<bucket_containing_lambda_zip> LambdaCodeZip=<packaged_lambda_zip>

In order to build a pipeline, we need to simulate the above steps.

This method can be used when the whole stack needs to be updated frequently, along with Lambda functions. If our Lambda function is dependent on SQS queue or the cloudwatch event needs update we could do this using a single pipeline.

Using AWS CDK

AWS CDK is infrastructure as code and supports Typescript, Javascript, Python, C# and Java.

Pre-requisites

  1. AWS CDK

Steps to deployment

  1. Package your code
  2. Upload it to S3 bucket
  3. Deploy the changes to the stack through aws cdk

There are 2 main differences using this method from the deployment using cloudformation template

  • Programming language constructs will help you add additional logic to your cdk code to extend your code and be creative with what you can do rather than just create AWS resources
  • Your build pipeline needs to contain Nodejs, your programming language runtime as well as the CDK tookit

Although CDK is powerful and extremely useful in a variety of scenarios, we would not recommend creating AWS Lambda pipelines this way. It is probably too complex for something as simple as continuously deploying changes to our Lambda code. There are too many variables and changes of your pipeline breaking down are higher.

For example,

  • Certain Nodejs versions (13.0.0 through 13.6.0) are not compatible with the AWS CDK
  • CDK libraries are updated quite often and constructs can become deprecated relatively quickly

Using AWS SAM

AWS Serverless Application Model (SAM) is a framework built by Amazon for building and deploying Serverless applications. If your Serverless application is more than just Lambda functions and provide functionality that needs databases, APIs and event source mappings, SAM makes your life easy to build and maintain these. You need the SAM CLI in order to create and maintain SAM-based applications. SAM also lets you locally build, test and debug applications.

It is important to note that SAM is an extension of AWS Cloudformation. Additional resources required can be defined in SAM template (which is also a cloudformation template)

Pre-requisites

  1. SAM CLI

Steps to deployment

  1. Create application (sam init)
  2. Build your application (sam build)
  3. Deploy your application (sam deploy –guided)
  4. Test your application locally (optional)

As you may have noticed, SAM not only makes it easier to create and deploy serverless applications, the framework includes best practices for serverless application creation, ability to test locally and simplified management of serverless application development.

Unless your Lambda functions is bare minimum, it is recommended to use a framework like SAM for Serverless application development

Using Serverless framework

The serverless application framework is a mature framework that existed long before AWS SAM came into being. The framework supports a variety of cloud providers and is simple to setup and use. It is intended to create and deploy FaaS (Function as a Service) and the support for additional AWS resources (like DynamoDB) may not be in-built.

Serverless also comes with a free Serverless Framework Dashboard account where you can monitor and test your service across cloud providers.

Pre-requisites

  1. Serverless CLI

Steps to deployment

  1. Define an endpoint in serverless.yml config
  2. Deploy your service (serverless deploy -v)
  3. Test your application (optional)

Serverless framework also supports testing your function/service locally.

Comparison of choices

FeaturesCloudformationAWS CDKAWS SAMServerless
Local TestingNoNoYesYes
MaturityRobust and matureOfficially in Beta.Still a young frameworkQuite mature
Code PlatformYAML/JSONMany programming languages (Typescript, Python, Java, etc.)Templates are in YAML, CLI used for maintenanceTemplates in YAML, CLI used for maintenance
SimplicitySimple but verboseSimple and compactA lot of generated bootstrap codeYes
DevOps/ Tooling SupportHighMinimalMinimalHigh
UsecasesSimple Lambda functions, extensively use CFM templatesUse CDK already or extensively use a programming language for infra automationCreate and maintain complex AWS Lambda functions along with Dynamo DB and API Gateway.Prefer simplicity and local testing, need DevOps or tooling capability. Need maturity in tooling

Overchoice

The intention is to introduce all the possible ways you can do serverless and pick what suits best for you. And for this reason, we have not delved into the details of individual deployment types. It is important to pick a methodology that suits your purpose, use-case and your environment.

At first, it looks like there are quite a few choices. Let us understand the differences and similarities better before we go ahead and pick one over another.

Whether you use CDK, SAM or Serverless, eventually all of these generate Cloudformation. However, both SAM and Serverless provide a layer of abstraction so that less code is required for serverless applications to be created. Unless your Lambda function is fairly simple and stand-alone and does not require a lot of additional AWS resources to be created, it makes sense to use a framework like SAM or Serverless for your Serverless development.

Also, if you have a collaborative development environment with a proper CI/CD pipeline, it makes sense to use a framework for development. This will make the development process more streamlined and less prone to errors.

If you are tightly integrated with AWS way of doing things (including AWS Devops) and need your serverless application to also create DynamoDB, API Gateway, SNS topic and other AWS resrouces, then SAM is the way to go. Serverless framework can help you create Serverless Functions on Azure and other cloud providers if that is what you want to do.

In our next blog, we will make a detailed analysis of SAM vs Serverless framework. Until then, enjoy your Serverless adventures!