IaC Solutions for AWS Cloud 2020: which one is the right for you?

A high-level graphic of how the IaC Tools are connected to AWS

I recently had the benefit of testing several tools to deploy Amazon Web Service(AWS) cloud infrastructure as code(IaC) in hands-on projects. Let me share my findings with you.

Just as a reminder: This guide focuses on the market leader AWS since it has the biggest variety in IaC frameworks. If your company is confident that AWS will be tamed by competitors and won’t disproportionately increase its prices in the next decade then you may stay with me and read this guide. If you are however using Azure or Google cloud or a mixture of all providers, you might as well skip this article and start learning terraform.

You probably know AWS’s user interface to interact with cloud services called AWS “console”. It is quiet useful to gain a visual representation of the service’s state, other static information and monitoring data.

On the other side manually manipulating the state of services (starting, stopping, updating) is also possible but gets time-consuming and unmanageable quickly for deployments of bigger applications.

Infrastructure as Code(IaC) or:

“…a single text-based description of a complete cloud infrastructure at a given point in time”

eliminates these problems by being versionable through a VCS like GIT and automatically deployable without human effort through a CI service such as CodePipeline. The open-source community brought up several solutions on how to manage IaC for your cloud resources:


AWS CloudFormation

The initial go-to solution for IaC in Amazon Web Services is called CloudFormation. A declarative specification that describes your whole cloud infrastructure in one or several .yaml files. You can use so-called “Stacks” to modularise your infrastructure deployment and will find various boilerplate templates of these stacks for almost any service provided by AWS on their website. Nowadays however it is very rare to see a devops engineer writing a “raw” CloudFormation script by hand due to some drawbacks explained below.

Since an example cloudFormation file for a basic deployment was too long to paste here, let me provide you with a link to a github repository with a good example

Pros

Cons

Unlike in programming languages where you have a clear distinction between language specific syntax and your own custom logic, CloudFormation-specific statements (i.e. “Parameters”, “Ref”) use the same syntax as attributes that act as definition for the respective services (i.e. “InstanceType” for an ec2 instance). That makes it very hard to distinguish patterns and find structure as a human reader.

Recommendation:

The core idea of CloudFormation is still very neat, however there are now multiple other solutions established in the open-source community and recognised by AWS that wrap away the complexity of cloudformation templates. Most importantly, using CloudFormation directly might entail security risks, simply because the complexity and repetitiveness of CloudFormation encourages human error. Therefore, it is a bold (and slightly sadistic) move to directly use CloudFormation in 2020 for managing your cloud infrastructure in code, except maybe for debugging. Consider using one of the following solutions instead

AWS Amplify

A Cli based framework to develop cloud-native Apps. Therefore rolling out cloud resources is just one of its many capabilities next to its main purpose of supporting the complete back to front development of a (mobile) app. The current definition of your infrastructure and locally adapted version is saved in a backend/ folder as Code and CloudFormation Files within your app repository. Similar to git it compares the two states once you hit the cli command “amplify push” and only deploys the changes.

A sample workflow using Amplify to add a lambda function to an existing project is available on the website. The CLI is a foolproof way to set up and connect services in the cloud:

amplify add function
? Provide a friendly name for your resource to be used as a label for this category in the project: lambdafunction
? Provide the AWS Lambda function name: lambdafunction
? Choose the function template that you want to use: (Use arrow keys)
❯ Hello world function
  CRUD function for Amazon DynamoDB table (Integration with Amazon API Gateway and Amazon DynamoDB)
  Serverless express function (Integration with Amazon API Gateway)

Pros

Cons

In general, at this point in time, you better hope that you can figure out the reason for any of the cli commands to fail since otherwise you might only be left with wiping the project and start afresh. Let’s be hopeful that the error handling of the CLI will improve here.

Recommendation

If you are a small or even one person team, rapidly developing a (mobile) App based on (AWS-) cloud resources and don’t know a whole lot about DevOps, Amplify is a very safe bet. I personally found it quiet useful and use it now preferredly for my personal projects. However, I am not sure if using AWS Amplify is a good idea in a corporate environment since you are encouraged to use the CLI-first which in turn is not versionable/traceable except for changes in the cloudformation templates.

While using Amplify, it served me well to get familiar with the contents of the backend/ folder to really understand what’s going on under the hood. I would advise you the same.

AWS SAM

SAM was the first IaC Framework for AWS i stumbled upon and introduced me to the magical wonders of IaC (Before i was still clicking around in the console). It is an extension to CloudFormation in which Identifiers for policies and the 3 fundamental services Lambda, DynamoDB and API Gateway were redefined in a much more lean and simple way, therefore reducing the lines of code in your template file substantially and improving your mental health while editing them. You can see that with a few lines of code you can define a DynamoDb Table HelloWorldTable, a REST endpoint at /helloworld, and a custom Lambda Handler printHelloWorldFunction:

Transform: AWS::Serverless-2016-10-31
Globals:
  Function:
    PermissionsBoundary: !Sub 'arn:${AWS::Partition}:iam::${AWS::AccountId}:policy/${AppId}-${AWS::Region}-PermissionsBoundary'

# Paramaters accessible in this template and your handlers
Parameters:
  AppId:
    Type: String
    Default: helloworld-app
  S3BuildBucket:
    Type: String
    Default: arn:aws:s3:::aws-sam-cli-managed-default-samclisourcebucket-13vlf6m4wtxxx

Resources:
  printHelloWorldFunction:
    Type: AWS::Serverless::Function
    Properties:
      # Path to folder of handlers
      CodeUri: ./src/handlers/
      # <fileName>.<mainFunction>
      Handler: printHelloWorld.printHelloWorldHandler
      Runtime: nodejs10.x
      MemorySize: 128
      Timeout: 60
      Description: A simple example includes a HTTP get method to get all items from a DynamoDB table.
      Policies:
        # Give Create/Read/Update/Delete Permissions to the SampleTable
        - DynamoDBCrudPolicy:
            TableName: !Ref HelloWorldTable
        - S3CrudPolicy:
            BucketName: !Ref S3BuildBucket
      Environment:
        Variables:
          # Make table name accessible as environment variable from function code during execution
          SAMPLE_TABLE: !Ref HelloWorldTable
      Events:
        # Here an Api endpoint is connected
        Api:
          Type: Api
          Properties:
            Path: /helloworld
            Method: GET
            
  # Creates a DynamoDB Table
  HelloWorldTable:
    Type: AWS::Serverless::SimpleTable
    TableName: HelloWorldTable
    Properties:
      PrimaryKey:
        Name: id
        Type: String
      ProvisionedThroughput:
        ReadCapacityUnits: 2
        WriteCapacityUnits: 2

Pros

Cons

Recommendation

I personally see the benefit of SAM as a compromise between Amplify and CloudFormation. There is practically no overhead for your development environment compared to a full-fledged App Framework like Amplify and at the same time it is structured way simpler than vanilla CloudFormation. If you are working on a basic 3-Tier Application and would only use Lambda, the API Gateway and a NoSQL database anyway you can confidently base your app on SAM. I also found it very easy to migrate the project from the REST-based API in SAM to a GraphQL-based API in Amplify by just copy-pasting the lambda handlers into the respective resolver functions for the graphQL API (if you ever need to).

AWS CDK

The AWS Cloud Development Kit (AWS CDK) is an open source software development framework to define your cloud application resources using familiar programming languages.You can see that using the CDK means a bit more boilerplate code but you can structure logic in your own desired way, like with the “addCorsOption” function.

You can see that using the CDK means a bit more boilerplate code but you can structure logic in your own desired way, like with the “addCorsOption” function:

import apigateway = require('@aws-cdk/aws-apigateway'); 
import dynamodb = require('@aws-cdk/aws-dynamodb');
import lambda = require('@aws-cdk/aws-lambda');
import cdk = require('@aws-cdk/core');

export class HelloWorldApp extends cdk.Stack {
  constructor(app: cdk.App, id: string) {
    super(app, id);

    const dynamoTable = new dynamodb.Table(this, 'items', {
      partitionKey: {
        name: 'id',
        type: dynamodb.AttributeType.STRING
      },
      tableName: 'HelloWorldTable',

      removalPolicy: cdk.RemovalPolicy.DESTROY, // NOT recommended for production code
    });

    const printHelloWorldFunction = new lambda.Function(this, 'printHelloWorldFunction', {
      code: new lambda.AssetCode('./src/handlers/'),
      // <Filename>
      handler: 'printHelloWorld',
      runtime: lambda.Runtime.NODEJS_10_X,
      environment: {
        TABLE_NAME: dynamoTable.tableName,
        PRIMARY_KEY: 'id'
      }
    });
    
    dynamoTable.grantReadWriteData(printHelloWorldFunction);

    const api = new apigateway.RestApi(this, 'helloWorldApi', {
      restApiName: 'HelloWorld Service'
    });

    const helloWorldEndpoint = api.root.addResource('helloWorld');
    const printHelloWorldIntegration = new apigateway.LambdaIntegration(printHelloWorldFunction);
    helloWorldEndpoint.addMethod('GET', printHelloWorldIntegration);
    addCorsOptions(helloWorldEndpoint);
  }
}

// An example how to include some custom CORS Definition programmatically
export function addCorsOptions(apiResource: apigateway.IResource) {
  apiResource.addMethod('OPTIONS', new apigateway.MockIntegration({
    integrationResponses: [{
      statusCode: '200',
      responseParameters: {
        'method.response.header.Access-Control-Allow-Headers': "'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token,X-Amz-User-Agent'",
        'method.response.header.Access-Control-Allow-Origin': "'*'",
        'method.response.header.Access-Control-Allow-Credentials': "'false'",
        'method.response.header.Access-Control-Allow-Methods': "'OPTIONS,GET,PUT,POST,DELETE'",
      },
    }],
    passthroughBehavior: apigateway.PassthroughBehavior.NEVER,
    requestTemplates: {
      "application/json": "{\"statusCode\": 200}"
    },
  }), {
    methodResponses: [{
      statusCode: '200',
      responseParameters: {
        'method.response.header.Access-Control-Allow-Headers': true,
        'method.response.header.Access-Control-Allow-Methods': true,
        'method.response.header.Access-Control-Allow-Credentials': true,
        'method.response.header.Access-Control-Allow-Origin': true,
      },  
    }]
  })
}

const app = new cdk.App();
new HelloWorldApp(app, 'HelloWorldApp');
app.synth();

Pros

Cons

Recommendation

AWS CDK is especially well-suited for teams from a programming background, projects with a need for highly customizable infrastructure that’s extending basic app development and fluctuating teams that need a human-friendly description of infrastructure for quick onboarding. Just as an example for the variety, there’s already an open-source extension of CDK for kubernetes “cdk8s” that can generate any kubernetes configuration .yaml files through a few lines of code. However, in my humble opininion, for simple app deployments(a.k.a “give me a database and Api”) this might be too much and you are better off with one of the above solutions (SAM and Amplify) and investing your time into developing the app first.

Terraform

Terraform is a tool for building, changing, and versioning infrastructure safely and efficiently. Terraform can manage existing and popular service providers as well as custom in-house solutions.A very lean example of how to define a lambda function with a dynamoDB Table without an API Gateway.

A very lean example of how to define a lambda function with a dynamoDB Table without an API Gateway:

provider "aws" {
   region = "us-east-1"
}

resource "aws_dynamodb_table" "HelloWorld" {
  name             = "HelloWorldTable"
  hash_key         = "id"
  billing_mode   = "PROVISIONED"
  read_capacity  = 5
  write_capacity = 5
  attribute {
    name = "id"
    type = "S"
  }
}

resource "aws_iam_role_policy" "lambda_policy" {
  name = "lambda_policy"
  role = aws_iam_role.role_for_LDC.id

  policy = file("policy.json")
}


resource "aws_iam_role" "role_for_LDC" {
  name = "myrole"
  # policy is stored externally 
  assume_role_policy = file("assume_role_policy.json")

}

resource "aws_lambda_function" "myLambda" {

  function_name = "func"
  s3_bucket     = "mybucket98745"
  s3_key        = "index.zip"
  role          = aws_iam_role.role_for_LDC.arn
  handler       = "printHelloWorldFunction.printHelloWorld"
  runtime       = "nodejs12.x"
}

Pros

Cons

Recommendation

The go-to solution for multi-cloud roll-outs or cloud providers apart from AWS. If you are already planning a project that might extend app development and will be based on multiple cloud providers, there is practically no alternative to terraform. You might not be too excited about learning another Syntax (HCL).. but who knows maybe you already used or will use other HashiCorp products in the future.

TL/DR

Of the mentioned services all have their individual pros & cons and deliver value in different scenarios. Let’s take in mind especially in the IT-World there is no such thing as a “perfect framework/paradigm”: just good enough solutions that hit the sweet spot between abstracting repetitive or complex stuff that you don’t want to deal with while giving you enough flexibility to change the stuff that you care about.

To wrap it up:

Sources

https://docs.microsoft.com/en-us/azure/devops/learn/what-is-infrastructure-as-code

https://zach-gollwitzer.medium.com/imperative-vs-declarative-programming-procedural-functional-and-oop-b03a53ba745c

https://www.bayesian.ninja/2019/11/aws-cloudformation-pros-and-cons.html

https://www.trustradius.com/products/terraform/reviews?qs=pros-and-cons