cloudformation

What every CIO should know about Cloud

Watch Angus and Allan talk about CIOs, Cloud and oranges or read the full transcript below.

What should every CIO know about Cloud, well, every CIO should know something about Cloud. I think we can agree on that part.

(laughing).

But, look, I don't have time for... I shouldn't put it that way. I think the era of the CIO that declared that we're a Cloud first business just because everybody else is doing that. I think that's finished and what we're seeing in Australia and you know from my experience internationally, as well, in some of the more advanced Cloud markets is, we're seeing people with technical understanding of what Cloud and other advances in technology can do for their business and that's what CIOs really need.

They need to understand how Cloud gives them and their businesses the freedom to be remarkable, not just we're gonna go there because everybody else is and that might deliver us some cost savings.

It's a variable topic, but I think that, you know. I think first of all, the move to, the move to Cloud is actually a, a move to make your business more nimble. Now I'm, that's the opportunity that lays in front of you. It's not always about cost savings. It's great that cost savings are a by product but that's exactly what it is, it's a by product of making a business more nimble.

I think, I think the agile model that, enables you to, you know to, to be more nimble. You, you need to be okay with value in some states and it's more about getting in there and trying uh, and doing some proof of concepts and you know, and just start building trust and start practicing the ways to make your business more nimble. It's not just about moving applications into, into any eco system.

It's always a great first step, but there has to be back in your mind. It has to be, actually you know, how do we transform this, how do we transform this business into being something that is going to adapt to oncoming competition.

Yeah, and the, the, the, the new generation of CIOs coming in. I think also need to realize that the partners that have got them to this point, can't get them to where they need to go next.

Yeah.

And, and it gets back to that conversation around the inter priority service delivery model. You can't get what you need from one or two big technology sourcing partners, I think. The best CIOs now understand that they need to engage specialists to deliver that transformation in their organisation. That's a very different perspective to what a CIO would have had in the past.

Yeah. Yeah, and I said if I bought apples today, 10 apples isn't gonna equal one orange. Like, it's like you could have a billion apples, you're not gonna get the orange they need, you know to be able to, at the level what they, they need to do.

I'm gonna have to get you to explain to me what that actually means later.

I don't know. I don't know. I just like apples. (laughing).

aidan-meyer-223547.jpg

AWS Security: Rule-based approvals for CloudFormation

2000px-AWS_Simple_Icons_AWS_Cloud.svg.png

AWS CloudFormation is a great tool for developers to provision AWS resources in a structured, repeatable way that also has the added benefit of making updates and teardowns far more reliable than if you were to do it with the individual resource APIs. It is the recommended approach for teams to utilise these stacks for the majority of their workloads and easily integrates into CI/CD pipelines.

Being so powerful, the complexity of CloudFormation templates can quickly become overwhelming and shortcuts or mistakes can occur. One common solution to this problem is to define a set of rules for the stack resources to be evaluated against. These rules can be generically defined, or specific to a particular teams needs. For example, a common issue that teams face is S3 buckets being incorrectly exposed. A rule may be defined that prevents this from occuring or notifies security teams.

Pre-Deploy vs. Post-Deploy Analysis

There are two distinct approaches to performing CloudFormation analysis; pre-deploy and post-deploy. Pre-deploy analysis reviews the content of the templates before they are created or updated whereas post-deploy analysis will look at the resultant state of the resources created/updated by the CloudFormation action.

Pre-deploy analysis will catch problems before they have a chance to manifest themselves in the environment. It is a more security-conscious approach but has the drawback of being significantly more difficult to predict or simulate the result.

Post-deploy has a much clearer picture of the state of resources and the result of the stacks actions, however some damage may already have been done the moment the resources are placed in this state. Amazon GuardDuty is a service which will alert on an Amazon-managed pre-defined set of rules on all resources within your account and is an example of a post-deploy analysis and alerting tool.

Validating templates before deployment

Let's discuss how a pre-deploy tool might work. The following example is written in Python 3:

def processItem(item):
  if isinstance(v, dict):
    for k, v in item.items():
      processItem(v)
  elif isinstance(v, list):
    for listitem in v:
      processItem(listitem)
  else:
    evaluate_ruleset(item)

processItem(template_as_object)

This is a very rudimentary evaluator that looks for all primitive types (strings, integers, booleans) within the template and evaluates against the ruleset.

Diving in

Consider the following template:

{
  "AWSTemplateFormatVersion" : "2010-09-09",
  "Description" : "A public bucket",
  "Resources" : {
    "S3Bucket" : {
      "Type" : "AWS::S3::Bucket",
      "Properties" : {
        "AccessControl" : "PublicRead"
      }
    }
  }
}

A rule that prevents S3 buckets from being publicly exposed may choose to interrogate the AccessControl property of any AWS::S3::Bucket resource for a public ACL and alert or deny based on that. This is how the majority of pre-deployment analysis pipelines work. Things can get tricky though when you involve the CloudFormation intrinsic functions, like Ref. Now consider the following template:

{
  "AWSTemplateFormatVersion" : "2010-09-09",
  "Description" : "A public bucket",
  "Resources" : {
    "S3Bucket" : {
      "Type" : "AWS::S3::Bucket",
      "Properties" : {
        "AccessControl" : { "Fn::Join" : [ "", [ "Publi", "cRead" ] ] }
      }
    }
  }
}

You'll quickly notice that even if a tool were to iterate through all properties in every Map and List, they would never find the "PublicRead" keyword intact. It's very common to join strings, or reference mappings in templates so a string-matching approach would be fairly ineffective.

CloudFormation resource specification

AWS produces a JSON-formatted file called the AWS CloudFormation Resource Specification. This file is a formal definition of all the possible resource types that CloudFormation can process. It includes all resources, their properties and information about those fields such as whether or not they need recreation when they are modified.

We can use this file to evaluate the properties for each resource and apply rulesets directly to individual properties, rather than the template as a whole. With logic around the processing of the intrinsic functions, we have created an open-source CloudFormation template simulator that is easily deployable in any environment. This simulator is able to evaluate most intrinsic functions like the example above and properly evaluate with our ruleset.

The template simulator can be found at https://github.com/KablamoOSS/cfn-analyse.

Thinking maliciously

Now consider the following template:

{
  "AWSTemplateFormatVersion" : "2010-09-09",
  "Description" : "A public bucket",
  "Resources" : {
    "TopicLower" : {
        "Type" : "AWS::SNS::Topic",
        "Properties" : {
            "TopicName" : "a-b-c-d-e-f-g-h-i-j-k-l-m-n-o-p-q-r-s-t-u-v-w-x-y-z-0-1-2-3-4-5-6-7-8-9"
        }
    },
    "TopicUpper" : {
        "Type" : "AWS::SNS::Topic",
        "Properties" : {
            "TopicName" : "A-B-C-D-E-F-G-H-I-J-K-L-M-N-O-P-Q-R-S-T-U-V-W-X-Y-Z"
        }
    },
    "S3Bucket" : {
      "Type" : "AWS::S3::Bucket",
      "Properties" : {
        "AccessControl" : { "Fn::Join" : [ "", [
            { "Fn::Select" : [ 15, { "Fn::Split" : [ "-", { "Fn::GetAtt" : [ "TopicUpper", "TopicName" ] } ] } ] },
            { "Fn::Select" : [ 20, { "Fn::Split" : [ "-", { "Fn::GetAtt" : [ "TopicLower", "TopicName" ] } ] } ] },
            { "Fn::Select" : [ 1, { "Fn::Split" : [ "-", { "Fn::GetAtt" : [ "TopicLower", "TopicName" ] } ] } ] },
            { "Fn::Select" : [ 11, { "Fn::Split" : [ "-", { "Fn::GetAtt" : [ "TopicLower", "TopicName" ] } ] } ] },
            { "Fn::Select" : [ 8, { "Fn::Split" : [ "-", { "Fn::GetAtt" : [ "TopicLower", "TopicName" ] } ] } ] },
            { "Fn::Select" : [ 2, { "Fn::Split" : [ "-", { "Fn::GetAtt" : [ "TopicLower", "TopicName" ] } ] } ] },
            { "Fn::Select" : [ 17, { "Fn::Split" : [ "-", { "Fn::GetAtt" : [ "TopicUpper", "TopicName" ] } ] } ] },
            { "Fn::Select" : [ 4, { "Fn::Split" : [ "-", { "Fn::GetAtt" : [ "TopicLower", "TopicName" ] } ] } ] },
            { "Fn::Select" : [ 0, { "Fn::Split" : [ "-", { "Fn::GetAtt" : [ "TopicLower", "TopicName" ] } ] } ] },
            { "Fn::Select" : [ 3, { "Fn::Split" : [ "-", { "Fn::GetAtt" : [ "TopicLower", "TopicName" ] } ] } ] }
        ] ] }
      }
    }
  }
}

The above template will actually evaluate to produce a public S3 bucket. This is because the S3 buckets "AccessControl" property uses characters from the "TopicName" attribute of the SNS topics. The format of these attributes is not formally documented in the CloudFormation resource specification, nor anywhere else. This means that there is currently no effective way to truly verify that resources with the intrinsic functions "Ref" or "Fn::GetAtt" are truly valid against the defined ruleset.

By the nature of CloudFormation, there isn't a perfect solution for static analysis which is why a multi-layered strategy is the most effective defense. If your organisation needs help protecting their environment, get in touch with us to find out how we can help you better protect yourself against these threats.