Lecture 160: Setting up a CI/CD pipeline for automated build and deploy

Let's recap the tools for building CI/CD pipelines:

• Jenkins: the granddaddy of CI/CD tools — powerful and highly configurable.
• GitHub Actions: built into GitHub, convenient and fast.
• GitLab CI/CD: a great pipeline tool if you're using GitLab.
• CircleCI: a cloud solution, easy to set up.
• Bitbucket Pipelines: the CI/CD tool in the Atlassian ecosystem.

For this lecture we'll use GitLab CI/CD — it's simple, visual, and works great with Docker.

Getting ready

Step 1: Create a repository on GitLab

First, make sure you already have a repository containing your Spring Boot app. If not, create a new repo and push your whole project there.

Step 2: Configure the .gitlab-ci.yml file

GitLab CI/CD uses the .gitlab-ci.yml file to define how the pipeline runs. This file should live in the root of the repository.

Creating the CI/CD pipeline

Step 1: Minimal .gitlab-ci.yml configuration

Let's start with a simple example that runs Java tests and builds the Spring Boot app:

stages: # Define pipeline stages
  - test
  - build

test-job: # Job for the test stage
  stage: test
  image: maven:3.8.6-jdk-11 # Use an image with Maven and JDK
  script:
    - mvn test # Run tests

build-job: # Job for the build stage
  stage: build
  image: maven:3.8.6-jdk-11
  script:
    - mvn package -DskipTests # Build the jar
  artifacts:
    paths:
      - target/*.jar # Save the jar as an artifact

• stages: define the execution order of the pipeline. First test, then build.
• image: specifies the Docker image that will be used to run the job. In our case it's Maven + JDK.
• script: the commands executed inside the job.

Step 2: Adding Docker to the pipeline

Now let's add a stage that builds a Docker image and pushes it to Docker Hub.

stages:
  - test
  - build
  - dockerize

dockerize-job:
  stage: dockerize
  image: docker:latest # Use the Docker image
  services:
    - docker:dind # Docker-in-Docker to manage containers
  before_script: # Log in to Docker Hub
    - echo "$DOCKER_PASSWORD" | docker login -u "$DOCKER_USERNAME" --password-stdin
  script:
    - docker build -t $DOCKER_USERNAME/my-spring-app:$CI_COMMIT_SHORT_SHA . # Build the image
    - docker push $DOCKER_USERNAME/my-spring-app:$CI_COMMIT_SHORT_SHA # Push the image

Note:

• services includes docker:dind so the GitLab Runner can run Docker commands.
• before_script is used for pre-job actions like logging into Docker Hub.
• $CI_COMMIT_SHORT_SHA is a variable containing the first 8 characters of the commit hash. It helps uniquely tag the image.

Step 3: Adding deploy to the cloud

Let's assume we deploy the app to a server over SSH. We'll add a deploy stage for that:

stages:
  - test
  - build
  - dockerize
  - deploy

deploy-job:
  stage: deploy
  image: ubuntu:latest
  before_script:
    - apt-get update && apt-get install -y ssh # Install the SSH client
  script:
    - ssh $DEPLOY_USER@$DEPLOY_HOST "docker pull $DOCKER_USERNAME/my-spring-app:$CI_COMMIT_SHORT_SHA && docker stop my-spring-app || true && docker rm my-spring-app || true && docker run -d --name my-spring-app -p 8080:8080 $DOCKER_USERNAME/my-spring-app:$CI_COMMIT_SHORT_SHA"

What happens here:

• We connect to the server over SSH.
• We pull the fresh Docker image of our app.
• We stop and remove the old container (if it was running).
• We start a new container.

Environment variables

For the pipeline to work we need to set some environment variables in GitLab:

• $DOCKER_USERNAME — your Docker Hub username.
• $DOCKER_PASSWORD — your Docker Hub password.
• $DEPLOY_USER — the SSH user for deployment.
• $DEPLOY_HOST — the host where the app will be deployed.

You can set them in Settings → CI/CD → Variables of your GitLab project.

Full .gitlab-ci.yml file

So our final file looks like this:

stages:
  - test
  - build
  - dockerize
  - deploy

test-job:
  stage: test
  image: maven:3.8.6-jdk-11
  script:
    - mvn test

build-job:
  stage: build
  image: maven:3.8.6-jdk-11
  script:
    - mvn package -DskipTests
  artifacts:
    paths:
      - target/*.jar

dockerize-job:
  stage: dockerize
  image: docker:latest
  services:
    - docker:dind
  before_script:
    - echo "$DOCKER_PASSWORD" | docker login -u "$DOCKER_USERNAME" --password-stdin
  script:
    - docker build -t $DOCKER_USERNAME/my-spring-app:$CI_COMMIT_SHORT_SHA .
    - docker push $DOCKER_USERNAME/my-spring-app:$CI_COMMIT_SHORT_SHA

deploy-job:
  stage: deploy
  image: ubuntu:latest
  before_script:
    - apt-get update && apt-get install -y ssh
  script:
    - ssh $DEPLOY_USER@$DEPLOY_HOST "docker pull $DOCKER_USERNAME/my-spring-app:$CI_COMMIT_SHORT_SHA && docker stop my-spring-app || true && docker rm my-spring-app || true && docker run -d --name my-spring-app -p 8080:8080 $DOCKER_USERNAME/my-spring-app:$CI_COMMIT_SHORT_SHA"

Debugging and common issues

• Docker-in-Docker problems If you see errors like cannot connect to the Docker daemon, make sure the docker:dind service is running and configured correctly.
• SSH not working Check that the SSH key is added to your server and that $DEPLOY_USER and $DEPLOY_HOST are set correctly.
• Problems with Docker Hub If Docker Hub returns an authentication error, verify that $DOCKER_USERNAME and $DOCKER_PASSWORD are set correctly.

You're all set! You just built a production-ready CI/CD pipeline for your Spring Boot app. Now every new feature will be automatically tested, built, and delivered to production while you sip your coffee.