A Faster Future: Leveraging AWS for DevOps
Updated: Aug 11, 2020
The future is fast. The life cycle of an application has gone from months to days. For DevOps teams, this dictates the necessity of several vital practices: migrating from monolithic architectures to microservices, developing CI/CD pipelines, and building infrastructure as code. AWS DevOps tools allow us to speed the process of development to delivery while maintaining these newly established best practices. Let’s take a look at how we can use AWS CodePipeline with CloudFormation. We’ll cover the steps for:
Connecting CodePipeline to Github as a source
Utilizing CodeBuild within a pipeline for testing/linting a Node.js application
Connecting our pipeline to GitHub for Source Control CloudFormation allows us to embrace infrastructure as code. In our CloudFormation template, we can define a CI/CD strategy with a highly adaptive CodePipeline, which pieces together our source code, CodeBuild projects, Lambda functions, and a CodeDeploy application/deployment group. Here’s a look at how we use GitHub integration when defining our CodePipeline in a CloudFormation template:
CodePipeline: Type: AWS::CodePipeline::Pipeline Properties: Name: Example-CodePipeline Stages: - Name: Source Actions: - Name: SourceAction ActionTypeId: Category: Source Owner: ThirdParty Provider: GitHub Version: 1 OutputArtifacts: - Name: SourceCodeOutputArtifact Configuration: Owner: Cedrus Repo: Demo Branch: Prod OAuthToken: !Ref GitHubToken RunOrder: 1
Now, when a branch is merged into the GitHub repo named Demo’s Prod branch, our pipeline will be triggered.
CodeBuild: Containerized Testing
CodeBuild is a very powerful tool for several reasons. It eliminates the need to provision, manage, and scale our own test/build servers. CodeBuild is fully managed, almost limitlessly scalable, and flexible for the needs of any given application. It also runs on-demand, saving money and resources. Here’s a look at how we can define a testing environment in a CloudFormation template. Our BuildSpec allows us to run commands during specific lifecycle hooks within a containerized environment that we define.
TestCodeBuildProject: Type: AWS::CodeBuild::Project Properties: Name: Sample-Test-Env Description: CodeBuild project for running Node.js Mocha tests Artifacts: Name: Test-Artifacts Type: CODEPIPELINE Packaging: NONE Source: Type: CODEPIPELINE BuildSpec: | version: 0.1 phases: install: commands: - npm install - npm install -g mocha pre_build: commands: - npm run lint build: commands: - npm test TimeoutInMinutes: 10 Environment: Type: LINUX_CONTAINER ComputeType: BUILD_GENERAL1_MEDIUM Image: aws/codebuild/nodejs:6.3.1
Along with unit tests, which will terminate the pipeline if failed, we are also able to check for any linting errors. This means best practices must be observed throughout the entire team. With internally defined rules, we are able to ensure our code is meeting strict quality standards even as the pace of development quickens.
Effective CI/CD pipelines are a key element in the fast paced world of DevOps. Here are a few things we learned while integrating CI/CD solutions with CloudFormation, CodePipeline, CodeBuild, and CodeDeploy.
GitHub integrates directly with CodePipeline, but BitBucket or other code sources require an extra step of configuration. For source code that exists outside of AWS or Github, we can configure a pipeline that uploads a compressed file of the code base to S3 (note that versioning must be enabled). CodePipeline can track this S3 object to trigger a pipeline when new code is uploaded.
Code as infrastructure makes it easy to move through environments from dev to test and production. CloudFormation is a powerful tool for creating predictable, tested, secure, repeatable environments.
CodePipeline allows for continuous delivery or continuous deployment depending on the needs of an application. The ability to add a step that requires human approval means that the whole process can be automated but deployment requires administrative consent before changes move to production.
For CI/CD to work well, there must be extensive test coverage. CodeBuild is a powerful tool, but it’s only as effective as the tests it runs. Robust unit/integration testing can eliminate any potential hiccups in deployment.
A little bit of configuration up front can save a lot of time and energy in the long run.
These methodologies have become best practices for a reason: using infrastructure as code, leveraging CI/CD pipelines, and embracing microservice and serverless solutions. These core concepts make development as fast as the future demands.