The CloudFormation Grain
The CloudFormation grain represents Torque's native support for AWS CloudFormation templates. Torque enables designers to utilize CloudFormation features for orchestrating both self-developed and community-driven CloudFormation modules in a consistent manner, making them accessible as reusable building blocks. For a complete YAML blueprint example, refer to Example 2: Webgame on S3 (using CloudFormation and Terraform).
Prerequisites
Before utilizing CloudFormation with Torque, ensure that you have the following prerequisites in place:
- An S3 bucket designated for the temporary storage of large templates.
For templates exceeding 50K bytes in size, Torque requires a "template-storage" location to upload templates from a Git repository, enabling the creation of CloudFormation stacks. To fulfill this requirement, you must either create a new S3 bucket or provide an existing one, which will serve as the landing area. Templates are fetched from Git and stored in this bucket, from where they can be launched.
- An AWS policy for use of cloudformation.
To grant Torque the necessary permissions to successfully provision resources, your credentials must include at least the following policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "BasicBucketOperations", // Sid for the template-storage bucket. If all your temaplates are smaller than 50K, this can be ommitted.
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:DeleteObject"
],
"Resource": [
"arn:aws:s3:::<your-bucket>/*" // the template-storage bucket name.
]
},
{
"Sid": "BasicCfnOperations",
"Effect": "Allow",
"Action": [
"cloudformation:CreateStack",
"cloudformation:DeleteStack",
"cloudformation:UpdateStack",
"cloudformation:DescribeStacks",
"cloudformation:DescribeStackEvents"
],
"Resource": "*"
},
{
// Any other permissions that are required to launch the resources inside the template. For example, if your template is using EC2 you can use:
//"Statement": [
// {
// "Action": "ec2:*",
// "Effect": "Allow",
// "Resource": "*"
// }
// ]
}
]
}
sourceโ
Please see the grain source for more details.
agentโ
agent
is now required for CloudFormation Grain. Please see the agent for more details.
authenticationโ
To enable Torque to connect to the AWS account and deploy the CloudFormation template, you have two options:
- Authenticate with Torque credentials (AWS access key and secret key OR AWS role ARN to be assumed by Torque)
- Authenticate with a service account that will be attached to the runner which provisions the infrastructure. The service account needs to be annotated by an AWS role ARN to be assumed by Torque.
Option 1:
grains:
database:
kind: cloudformation
spec:
source:
path: github.com/org/repo.git//cloudformation/rds
...
agent:
name: my-agent
authentication:
- credential_name or {{.inputs.credentials_input_name}}
Option 2:
grains:
database:
kind: cloudformation
spec:
source:
path: github.com/org/repo.git//cloudformation/rds
...
agent:
name: my-agent
service-account: agent-service-account # Optional. If not provided, Torque will try to use the default service account of the agent.
Template-Storage Configurationโ
In the CloudFormation grain, you will need to specify the details of the S3 bucket serving as the template-storage for larger templates.
grains:
database:
kind: cloudformation
spec:
...
template-storage:
bucket-name: <name> # required, can be templated
region: <region> # required, can be templated
key-prefix: <prefix> # optional, can be templated . This is the file path where the template will be located inside the bucket.
The template-storage is an optional element in the grain, however if you will not provide it only templates which are smaller than 50K bytes will be supported.
inputsโโ
Similar to blueprint inputs, CloudFormation grain inputs allow you to reuse the same CloudFormation module in different ways. Inputs provided to the CloudFormation grain are used when launching the CloudFormation module.
tagsโโ
Whenever a CloudFormation grain is launched, all resources created during the deployment process are automatically tagged with Torque's system tags, built-in tags and custom tags.
outputsโโ
Outputs are strings generated by CloudFormation during the deployment process.
grains:
database:
kind: cloudformation
spec:
source:
path: github.com/org/repo.git//cloudformation/rds
...
authentication:
...
outputs:
- agent_name
- connection_string
Exampleโ
grains:
CFN-S3-Bucket:
kind: cloudformation
spec:
source:
path: https://.../AWSS3Bucket.yaml
inputs:
- AccessControl: '{{ .inputs.["Access Control"] }}'
- BucketName: '{{ .inputs.["Bucket Name"] }}-{{ sandboxid | downcase }}'
outputs:
- Arn
- DomainName
Note about reconciling an drifted Cloudformation grainโ
Resolving drift in AWS CloudFormation involves acknowledging the updated configuration as the intended state and adjusting the stack template accordingly. Conversely, in Torque, the process of drift resolution or reconciliation entails undoing changes made to cloud resources and restoring them to the original template. It is important to understand this distinction before reconciling CloudFormation grains.