Introduction
Github actions is one of the latest features added on Github, which allows developers to automate and execute workflows within their repository. Until now, Github has been the place where we committed our code and worked with our teams to improve our software. Pull requests, code reviews and bug issues have been our core activities, but many times we relied on external tools to implement continuous integration (CI) and continuous development (CD). However, those days are gone and Github actions allow us to implement CI/CD practises within our repository, minimizing the effort and complexity we would normally have.
Before we start, let's start explaining what is CI/CD.
Continuous Integration
Continuous Integration (CI) is a software practise that checks how a change in the codebase integrates with the rest of the software.
Everytime we need to implement a new feature, we create a branch and commit our changes within that branch. When we finish, we raise a pull request and wait from other developers to review the changes and merge it in master. However, requesting from other developers to review our code isn't the only thing that we should be doing. We need to verify that our changes won't affect the rest of the software, so we need to run a few verification tests. Those tests include tasks such as:
- Unit tests
- Functional Tests
- Lint and Formatting
- Code analysis
- Code coverage
- Custom checks
These tests are running on every commit, and if our changes pass those tests, it means that we can trust them. In addition, having someone else verifying that your changes are okay, it gives more confidence and credibility. Last but not least, if you are working in a team where multiple developers are working on different parts of the software, those tests will warn you if anything has changed since the last time you worked on a certain part of the software.
Continuous Development
Continuous deployment (CD) is a process that automates the deployment cycle of a software, meaning that deployments occured frequently in smaller chunks
While CI focuses on running tasks that verify the reliability of your software, continous development extends this idea and automates the deployment process of the software. This means that at a certain part in your pipeline you have automated the release of the software, allowing to work more agile. Now, software is released more frequently, in smaller chunks and iteratively.
Together, Continuous Integration and Continuous Development form a development process known as continuous software development (CSD).
Github Actions
With Github actions we can build our own workflows from our own repository, without us needing to set up anything. A workflow is running on a machine hosted from Github and the only thing we need to do is to provide the configuration of the workflow. The configuration is specified using a YAML file within the .github/workflows directory.
You heard me repeating the word workflow multiple times, but what actually it is?
Workflow is a process running in your repository, which may include multiple jobs running in parallel, or even sequentially. A workflow listens to events, which are things that occur in your repository. Some of the most common events are:
- Pull Request
- Push
- Issue
- Comment
For example, a pull request event is triggered whenever you raise a PR, which triggers a job to run tasks such as unit tests, formatting, lint etc.
With that being said, a job is a container of multiple steps, which are triggered in order, and by default depend on the status code of the previous steps. A job contains at least one step.
A step is a command or action in a particular step. This is the smallest task executed within a workflow, nonetheless, still very powerful. A simple step is to run the unit tests.
In addition, github actions allow you to share functionality, known as actions. For example, there are actions available to the public that execute a certain type of functionality. These actions can be re-use by others, avoiding re-writing things from scratch.
Demo
We talked a lot, but now its time to demonstrate an implementation of CI/CD using gitbhub actions.
The project includes a simple API with a single endpoint /hello/:name that returns a Hello [name] message to everyone who call it. The focus in this example is not to flex our skills on how we build services, rather than to understand:
- the meaning of CI/CD practises
- how github actions can help to simplify our development experience.
The code is available in this repository.
Below, we have set a pipeline which runs on a PR against master. The workflow includes a single job, with multiple steps within it. At the beginning, we checkout from the repository and install the go runtime in the ubuntu-20.04 machine. We check the version and then run vet, which analyses our code and tries to find faulty segments within our code. If vet is successful, we run the unit tests and store the coverage value in an output. We use this syntax ::set-output name=coverage::$(command) to store the output. We need the output so we can re-use it in the following step, where we compare it with a baseline value. If everything is successful, we exit with a status code of 0, otherwise we return a status code 0f 1 and the pipeline fails.
name: PR Pipeline
on:
pull_request:
branches:
- master
jobs:
pr:
name: Pull request
runs-on: ubuntu-20.04
timeout-minutes: 5
steps:
- uses: actions/checkout@v2
- uses: actions/setup-go@v2
with:
go-version: '^1.16.2'
- name: Tools versions
run: |
go version
- name: Run Vet
run: |
make vet
- name: Run Unit Tests
id: unit
run: |
echo "::set-output name=coverage::$(make unit | egrep -o '[0-9]+\.[0-9]+%' | egrep -o '[0-9]+\.[0-9]+')"
- name: Run Code Coverage
env:
baseline_coverage: 85.0
coverage: ${{ steps.unit.outputs.coverage }}
run: |
is_greater=$(echo "$coverage $baeline_coverage" | awk 'BEGIN { print ($1 >= $2) ? "0" : "1" }')
exit $is_greaterHere are a few screenshots of the output after you run the pipeline in Github:
The following file is the master pipeline and runs on every push in master. This workflow includes two jobs, the test and deployment jobs. The deployment has a dependency on the test job, and this is specified using the needs: [test] syntax. By default the jobs are running in parallel, however, setting this dependency means that they are running sequentially now. The test job includes the same steps we executed in the PR, so let's focus on the deployment job.
It's worth mentioning that each job runs on a separate machine, meaning that we need to re-install our dependencies on each job. This is one of the criteria I personally use to decide if I want things to run in multiple steps or jobs.
name: Master Pipeline
on:
push:
branches:
- master
jobs:
test:
name: Test pipeline
runs-on: ubuntu-20.04
timeout-minutes: 5
steps:
- uses: actions/checkout@v2
- uses: actions/setup-go@v2
with:
go-version: '^1.16.2'
- name: Tools versions
run: |
go version
- name: Run Vet
run: |
make vet
- name: Run Unit Tests
id: unit
run: |
echo "::set-output name=coverage::$(make unit | egrep -o '[0-9]+\.[0-9]+%' | egrep -o '[0-9]+\.[0-9]+')"
- name: Run Code Coverage
env:
baseline_coverage: 85.0
coverage: ${{ steps.unit.outputs.coverage }}
run: |
is_greater=$(echo "$coverage $baeline_coverage" | awk 'BEGIN { print ($1 >= $2) ? "0" : "1" }')
exit $is_greater
deployment:
name: Deployment
runs-on: ubuntu-20.04
needs: [test]
timeout-minutes: 10
steps:
- name: Checkout Repository
uses: actions/checkout@v2
- name: Configure aws credentials
uses: aws-actions/configure-aws-credentials@v1
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: us-east-1
- name: Login to Amazon ECR
uses: aws-actions/amazon-ecr-login@v1
- name: Build Docker image
id: docker_build_step
env:
IMAGE_TAG: ${{ secrets.ECR_REGISTRY }}:${{ github.sha }}
run: |
docker build -t hello:latest -f Dockerfile .
docker tag hello:latest ${IMAGE_TAG}
docker push ${IMAGE_TAG}
echo "::set-output name=image_tag::${IMAGE_TAG}"
- name: Update Task Definition
uses: aws-actions/amazon-ecs-render-task-definition@v1
id: task_definition_step
with:
task-definition: ./deployments/taskDefinition.json
container-name: hello
image: ${{ steps.docker_build_step.outputs.image_tag }}
- name: Deploy Amazon ECS task definition
uses: aws-actions/amazon-ecs-deploy-task-definition@v1
with:
task-definition: ${{ steps.task_definition_step.outputs.task-definition }}
service: hello-service
cluster: cluster
wait-for-service-stability: true
As the name suggests, the deployment job takes care of the deployment of the software. The app is hosted in AWS and its running in an ECS cluster. The cluster is running with Fargate, which is the serverless implementation of containers for AWS. In addition, we use Elastic Container Registry (ECR) to push our image, which is the location an ECS service uses to find a certain version of the application.
We run these steps in the deployment job:
1) Checkout our repository
2) Configure our AWS credentials, meaning that update the credentials in ~/.aws/credentials. Pay attention that the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY are stored as secrets. We don't want them to be available to others. We use the ${{ secrets.[name] }} syntax to access a secret from Github.
3) Build, tag our image and push it in the registry. We use the commit sha to version our image. We need the image tag in the task definition step, so we set it as an output.
4) Update the task definition of the application. If you are not familiar with AWS, a task definition includes the information needed to run an application in ECS. This definition is used by ECS service to run a task.
5) Deploy the application in our cluster and wait until the job reports a successful code.
Here are a couple of screenshots of the pipeline output:
Setting those two workflows, we managed to implement CI/CD practises and improve the development experience in our repository. We managed to verify programmatically that our changes in the repository won't break previous implementations of the app, and at the same time release our software without any human intervention.
Summary
- Continuous Integration (CI) is a common practise used in development to verify that changes dont't break previous implementation of the apps. It usually includes testing, lint, formatting, code analysis etc.
- Continuous Development (CD) extends the idea of CI and automates the release process of a software. It also sets the foundations for agile development.
- Github Actions is one of the latest features added in Github that allows you to run workflows from your repository. A workflow listens to events and executes jobs. A job is a container of multiple jobs/actions.
