With our microservices deployed on Kubernetes, the next step is to automate the process of building, testing, and deploying updates. Continuous Integration and Continuous Deployment (CI/CD) pipelines streamline these workflows, ensuring consistent and reliable deployments while reducing manual intervention.
In this guide, we’ll:
Set up a GitLab CI/CD pipeline.
Automate Docker image builds and pushes.
Deploy microservices to Kubernetes directly from GitLab.
Incorporate best practices, including testing, environment separation, and rollback mechanisms.
By the end of this part, you will have a fully functional CI/CD pipeline integrated with GitLab that automates the deployment of your microservices to Kubernetes.
Step 1: Understanding CI/CD in GitLab
1.1 What is CI/CD?
Continuous Integration (CI) automates the process of integrating code changes into the main branch and running tests to validate them.
Continuous Deployment (CD) automates the process of deploying these validated changes to production or staging environments.
Why GitLab?
Built-in Docker Registry: GitLab provides a container registry for storing Docker images.
Kubernetes Integration: GitLab supports direct integration with Kubernetes clusters.
YAML Configuration: CI/CD pipelines are defined in a .gitlab-ci.yml file, offering flexibility and version control.
Step 2: Prerequisites
Before creating the pipeline, ensure the following are set up:
2.1 GitLab Repository
Your codebase should be hosted in a GitLab repository. If not, create one and push your code:
git init
git remote add origin <gitlab-repository-url>
git push -u origin main
2.2 Kubernetes Cluster
Ensure the Kubernetes cluster from Part 3 is up and running.
Install the GitLab Kubernetes Agent to enable deployment:
kubectl apply -f https://gitlab.com/gitlab-org/cluster-integration/cluster-agent/-/raw/main/manifests/agentk.yaml
2.3 GitLab Container Registry
- Enable the GitLab Container Registry for your repository under Settings > Packages and Registries > Container Registry.
2.4 GitLab Personal Access Token
- Create a personal access token in GitLab with
api and write_registry scopes. Use this for authentication in the pipeline.
Step 3: Writing the .gitlab-ci.yml File
The .gitlab-ci.yml file defines all stages of the pipeline. We’ll break it into stages: Build, Test, and Deploy.
3.1 Build Stage
The Build stage creates Docker images for the microservices and pushes them to the GitLab Container Registry.
YAML Configuration:
stages:
- build
- test
- deploy
variables:
DOCKER_REGISTRY: registry.gitlab.com
PROJECT_PATH: your-namespace/your-project
KUBECONFIG: /kubeconfig/config
build_user_service:
stage: build
image: docker:latest
services:
- docker:dind
variables:
IMAGE_TAG: $CI_REGISTRY_IMAGE/user-service:$CI_COMMIT_SHORT_SHA
script:
- docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $DOCKER_REGISTRY
- docker build -t $IMAGE_TAG ./user_service
- docker push $IMAGE_TAG
only:
- main
Explanation:
Repeat similar steps for other microservices, modifying paths as necessary.
3.2 Test Stage
Automate tests to ensure the integrity of the code before deployment.
YAML Configuration:
test_user_service:
stage: test
image: python:3.10
script:
- pip install -r ./user_service/requirements.txt
- pytest ./user_service/tests
only:
- main
Explanation:
Uses the official Python image.
Installs dependencies and runs tests using pytest.
Repeat for other microservices by specifying their respective paths.
3.3 Deploy Stage
Deploy Docker images to the Kubernetes cluster using kubectl.
YAML Configuration:
deploy_user_service:
stage: deploy
image: bitnami/kubectl:latest
variables:
IMAGE_TAG: $CI_REGISTRY_IMAGE/user-service:$CI_COMMIT_SHORT_SHA
before_script:
- mkdir -p /kubeconfig
- echo "$KUBECONFIG_CONTENT" | base64 -d > $KUBECONFIG
script:
- kubectl apply -f ./k8s/user-service-deployment.yaml
- kubectl set image deployment/user-service user-service=$IMAGE_TAG
only:
- main
Explanation:
Step 4: Managing Environments
4.1 Environment-Specific Configurations
Create separate environments for staging and production with different configurations.
Example YAML:
deploy_to_staging:
stage: deploy
script:
- kubectl apply -f ./k8s/staging/user-service-deployment.yaml
deploy_to_production:
stage: deploy
script:
- kubectl apply -f ./k8s/production/user-service-deployment.yaml
only:
- tags
Best Practices:
Feature Branch Deployments:
Environment Variables:
- Store environment-specific variables securely in GitLab Settings > CI/CD > Variables.
Step 5: Adding Rollback Mechanisms
Ensure deployments can roll back to the previous stable version in case of failure.
Rollback Script:
rollback_user_service:
stage: deploy
script:
- kubectl rollout undo deployment/user-service
when: on_failure
Best Practices:
Step 6: Monitoring the Pipeline
GitLab provides a visual dashboard to monitor pipeline status.
Navigate to CI/CD > Pipelines in your GitLab repository.
View logs and details for each job by clicking on its name.
Identify and fix issues in failed jobs.
Step 7: Best Practices for GitLab CI/CD
Branch Protection:
- Restrict deployments to the
main branch or specific tags.
Artifact Management:
Caching:
Conclusion
Now you have set up a CI/CD pipeline in GitLab that:
Builds Docker images for your microservices.
Runs automated tests to validate changes.
Deploys updates to a Kubernetes cluster.
This pipeline ensures reliable, consistent deployments, saving time and reducing errors.
In Part 5, we will focus on advanced topics, including database sharding, scaling strategies, and integrating observability tools like Prometheus and Grafana to monitor the health and performance of your microservices.
Happy Deployment!
