Hands-On Auto DevOps with GitLab CI

To speed up our build pipeline while still maintaining a high-quality test infrastructure, we need to know how to run multiple jobs in parallel.

• Create multiple jobs in a single stage

• Observe jobs running in parallel

• Create multiple identical job instances and run them in parallel

Downloading dependencies has the potential to slow down our build. If we use caching, it can speed things up.

• Configure our build to cache downloaded dependencies

• Configure our build to cache only certain directories

• Create separate caches for separate builds and separate branches

We need to capture outputs from our build process so we can analyze the result of the build.

• Declare artifacts in our build

• Capture artifacts even from failed builds

• Configure artifacts to expire after one week

Artifacts can be used to pass data between jobs in different stages. This makes it easier to divide our build up into multiple jobs.

• Create a build pipeline with multiple stages and artifacts defined

• Customize which jobs receive which artifacts

• Observe how artifacts are provided in default and customized cases

We need to provide private information to our jobs without exposing that information in our repository.

• Declare variables at the project level

• Declare variables at the group level

• Consume project and group variables in the build

To track deployments from our builds, we need to organize our deployments into environments.

• Declare an environment for a build job

• View an environment and its deployed application

• View the deployment history for an environment

To deploy an application to Kubernetes, we need to understand core Kubernetes resources.

• Create a service and deployment for our web application

• Create a service and deployment for our database

• Create persistent storage and web traffic routing resources

To deploy our application to Kubernetes, we need to pass Kubernetes configuration and authentication to our deployment job.

• Associate a Kubernetes cluster with the environments in our project

• Use passed through Kubernetes credentials

• Deploy our application from our GitLab CI deployment job

Kubernetes needs credentials to pull our custom Docker image from our project’s GitLab Container Registry.

• Create a registry deploy token in GitLab

• Create environment variables with our deploy token credentials

• Create and use a Kubernetes image pull secret

To review changes to our application before we deploy them to production, we need to be able to run multiple versions of our application in the same Kubernetes cluster.

• Configure dynamic environments for non-production deployments

• Use a unique naming convention for our Kubernetes resources

• Create a feature branch change and use GitLab CI Review Apps to test it

Integration tests have external service dependencies like databases that we need to have available during the build.

• Declare a database service in our test job

• Pass environment variables to our database service

• Configure a service alias and use it to access the database from our test

To keep our code quality high, we need to automatically monitor it for every potential change we review.

• Create an automated code quality job

• Create a code change and a merge request

• Observe the code quality results for the code change in the merge request

Some security testing can only be accomplished when our application is deployed. We can use GitLab Review Apps to perform dynamic security scanning.

• Add a security scanning job for feature branches

• Create a feature branch and merge request

• Use the security scan detailed report and summary

To monitor our application, we need it to continually produce metrics that can be collected by a monitoring application.

• Add a Prometheus client library to our Node.js application

• Configure a metrics endpoint that reports current data

• Run our application and see the raw metrics from the endpoint

Prometheus is a popular monitoring application that integrates well with Kubernetes. GitLab can install it into our Kubernetes cluster automatically.

• Configure our application’s service for scraping by Prometheus

• Deploy our application and observe its metrics in the Prometheus dashboard

• See application metrics from Prometheus right in our GitLab project website

If we need custom tools or configuration for an automated build, we need to create a dedicated runner for our project.

• Install GitLab Runner

• Register our runner with our GitLab project

• Use our dedicated runner to run a build

To have proper control over our custom CI infrastructure, we need deployment automation so we can create CI infrastructure automatically.

• Provision virtual machines in AWS using Ansible

• Install and configure GitLab Runner automatically using Ansible

• Use our new runner to run a build in our GitLab CI project

To support many projects and builds in our custom CI infrastructure, we need a way to scale runners up or down automatically.

• Use Ansible to provision a Docker Machine runner

• Configure our Docker Machine runner with AWS credentials

• Run a build and observe our build environment scale to execute it

Because we have a Kubernetes cluster already configured for deploying our application, we can also use it to run builds.

• Install GitLab Runner on our Kubernetes cluster

• Observe the GitLab Runner in our Kubernetes cluster

• Start a build and observe it running in Kubernetes

To fully move to custom CI infrastructure, we need to run our own instance of GitLab Server.

• Install and configure GitLab Server using Ansible

• Configure Ansible variables to allow a dedicated runner to attach to our GitLab server

• Install and configure GitLab Runner using Ansible and see it in our private GitLab server