Gitlab Runner and Maven – Guide [With the efficient cache method]

Gitlab Runner and Maven it’s the tools that we need to automate our build process to release any Java application. We will learn everything that is required to configure Gitlab and Runner to build our application correctly.

Gitlab is a Git repository manager with great CI/CD integration, without managing many plugins like Jenkins. And the fact that we are already close to the source code makes the process easier. We can visualize code and the build process a the same place.

Also, Gitlab provides us the ability to create multiple Runners, which is the place where the build happens effectively, not on the Gitlab server itself.

In addition, all processes behind the scene for a build on Gitlab happens inside a Docker container. It means that we don’t need to install any extra software on the Runner.

There are two critical files that we need to pay attention to and focus on to build our Java application using Maven on Gitlab. Let’s see each of them:

• .gitlab-ci.yml

• settings.xml

Gitlab CI YML

What is gitlab-ci.yml?

The .gitlab-ci.yml is the file that we configure our project on Gitlab and specify all the steps to build our application.

It’s a YAML file, so it’s straightforward to read. And we need to follow the attributes that Gitlab understands. Also, we can specify the commands that we would execute to build the application through a Shell Script or command line on that file. So, if you already have in mind all processes that you should follow to compile the application, it will be easier and faster to complete all steps on Gitlab.

Gitlab CI Java Maven Example

So, let suppose that to build your Java application using Maven, you need to run one of the commands below:

mvn package -U

or

mvn deploy

build:
  stage: build
  script:
    - mvn package -U

Let’s see the result:

Running with gitlab-runner 13.11.0 (7f7a4bb0)
  on gitlab-runner-maven B-Scbyv_
Preparing the "docker" executor
00:03
Using Docker executor with image alpine ...
Using locally found image version due to "if-not-present" pull policy
Using docker image sha256:d4ff818577bc193b309b355b02ebc9220427090057b54a59e73b79bdfe139b83 for alpine with digest alpine@sha256:adab3844f497ab9171f070d4cae4114b5aec565ac772e2f2579405b78be67c96 ...
Preparing environment
00:00
Running on runner-n-ubbyq-project-546-concurrent-0 via ip-192-168-15-25...
Getting source from Git repository
00:02
Fetching changes with git depth set to 50...
Reinitialized existing Git repository in /builds/gitlab/runnner/maven/docker/.git
Checking out be7dafc1 as temp2...
Executing "step_script" stage of the job script
00:01
Using docker image sha256:d4ff818577bc193b309b355b02ebc9220427090057b54a59e73b79bdfe139b83 for alpine with digest alpine@sha256:adab3844f497ab9171f070d4cae4114b5aec565ac772e2f2579405b78be67c96 ...
$ mvn package -U
/bin/sh: eval: line 104: mvn: not found
Cleaning up file based variables
00:01
ERROR: Job failed: exit code 127

Gitlab CI Maven Not Working: Maven Not Found

But, wait. We know that we don’t have the Maven installed on Runner. But, as I mentioned before, we rely on the Docker container to build our application, so there is no limitation about dependencies that we should install and configure to use it.

However, to build your Java application using Maven on Gitlab, we need to specify a Docker Image on the .gitlab-ci.yml file so the Docker container will execute all your script and steps within it. Otherwise, you will get the error message like the above: “/bin/sh: eval: line 104: mvn: not found.”

Gitlab CI Maven and Docker: How do I find the right Docker Image for me?

The Maven has many releases, and in my experience doesn’t matter which version of Maven you use. You will always find all versions on the Docker Hub explorer.

On Docker Hub explorer, you can navigate through the Tags session and filter by a specific version.

Gitlab with Maven using HTTPS:

I would like to remember that if you pick up the Maven greater than or equal to 3.8.1, you must use HTTPS when you specify the host and address for your repository. So, if you have any issues regarding the certificate, you can use an older version and then use HTTP only. Also, all processes run inside the Docker, so the Maven choice will not impact the Gitlab pipeline.

Gitlab CI CD Maven Example

Let’s see our example.

image: maven:3.6-jdk-8
variables:
  MAVEN_CLI_OPTS: "-s .m2/settings.xml --batch-mode"
  MAVEN_OPTS: "-Dmaven.repo.local=.m2/repository -Dmaven.artifact.threads=50"
cache:
  paths:
    - .m2/repository/
build:
  stage: build
  script:
    - mvn clean $MAVEN_CLI_OPTS $MAVEN_OPTS deploy -DskipTests

In our example, we are using the maven:3.6-jdk-8 from the official Maven repository within the Docker Hub, and we choose the tag/version 3.6-jdk-8. Also, we used the Gitlab variables to specify the parameters that are required to execute our build.

However, sometimes we need to build our own Docker image, to add some dependencies that the application requires to perform the build process. For example, your Maven project may have a different code language, like a Typescript for your UI using Angular, so in this case, you will need to add the nodejs within your image beside the Maven.

In this case, you can generate a Docker Image using the official Maven as a based image and add a new layer by installing the nodejs on it. Later, you need to push the new Docker image to the Docker Hub or any other Repository, like ECR from AWS. Just remember that the Runner should have access to the registry from where you should pull the images.

Gitlab Set Maven settings.xml

Now let’s see our second important file: settings.xml.

There is a minimum specification for this file, and will show an example:

<settings>
    <profiles>
        <profile>
            <id>default-profile</id>
            <activation>
                <activeByDefault>true</activeByDefault>
            </activation>
            <properties></properties>
            <repositories>
                <repository>
                    <id>release-repo</id>
                    <url>http://gitlab-runner-maven/repository/repository-name/</url>
                    <releases>
                        <enabled>true</enabled>
                    </releases>
                </repository>
            </repositories>
            <pluginRepositories>
                <pluginRepository>
                    <id>release-repo</id>
                    <url>http://gitlab-runner-maven/repository/repository-name/</url>
                    <releases>
                        <enabled>true</enabled>
                    </releases>
                    <snapshots>
                        <enabled>true</enabled>
                    </snapshots>
                </pluginRepository>
            </pluginRepositories>
        </profile>
    </profiles>
    <servers>
        <server>
            <id>release-repo</id>
            <username>user</username>
            <password>pass</password>
        </server>
    </servers>
</settings>

This file’s vital because when the Maven from within the Docker container tries to build your application on the Runner, the Maven needs to know which Maven repository to use.

Notice that we also specify the password to authenticate on the Maven repository to get access to upload and deploy our artifacts. So, for example, you can use the Gitlab CI to download artifacts from Nexus when you specify your Nexus Server host on the settings.xml.

Gitlab CI CD Tips

The approach above works well for small projects and small organizations, it also is the most straightforward way to get started with your first project and learn how the CI/CD from Gitlab works. However, for a big organization and when we have several projects on Gitlab, this approach has some issues:

 1 – Security: The settings.xml contains the password from your repository, and any person with access to the project could read it.

Tip: Alternatively, we can use the Variables on Gitlab from the Pipeline Configuration, where we can add environment variables, mask and encrypt it. But, this requires adding the variable for each project.

2- Maintainability: This approach is not sustainable in the long term. For example, suppose you have your Nexus server or any other Maven Repository. One day, you decide to change the user or password or even the host/address from the repository or any other configuration. In that case, you need to go through all your projects and change all files manually.

3- Cache: is not efficient. It’s common for big projects to have multiple small or big applications that share the same dependencies. So, it’s not worth building our cache for each project in this scenario, as we did in the example above. 

Gitlab Speed up Build

Let’s see how we can improve our build process to build our Java application using Maven on Gitlab.

Because all our builds happen within the Runner, it makes more sense to centralize all configurations regarding the build process and cache inside the Runner. So, we don’t need to change all projects if needed.

Usually, the Developers or the regular Gitlab user doesn’t have access to the SSH in the Runner. So, for example, we can safely store the password on the settings.xml file.

So, How it works? How do we share this configuration between all projects?

Let’s see first how Gitlab interacts with the Runner.

Install and Register Gitlab Runner

When your project has specified an Image to run your Job (build), Gitlab contacts the Runner server using the Gitlab API (it uses protocol HTTPS) and asks it to create a Docker container to execute your task within it.

Also, it’s worth remembering that this happens for each build/pipeline. We never reuse the same Container to build our application one second time. So, in the end, the Runner always kills and destroys the Container.

If you are using Fargate Runner, check out the post here.

Gitlab Share File Between Projects

You may be thinking, how to persist a configuration file and share it with all your projects without adding any file to your project?

We can leverage the Docker volumes for that proposal. But, to make it work, we need to configure our Runner to allow the Docker container to share volumes. And this is done when you register the Runner to the Gitlab server.

In other words, the Runner will hold the files and share those files for every new Container created to execute one Job to the Gitlab.

Note: If your Runner was deployed using AWS Fargate, the only way to cache is using S3 Bucket. Check for more information about the AWS Fargate Runner.

See the example, using the gitlab-runner binary on your Runner server:

gitlab-runner docker volumes

gitlab-runner register -r  -name gitlab-runner-maven -u  -n --executor docker --docker-image alpine --docker-volumes "/root/.m2:/root/.m2"

Also, we can specify the local repository to use as our cache and speed up our build process. 

We know that the default folder for the local repository on Maven is always ~/.m2/repository, and the settings.xml is also inside of ~/.m2/repository

If you choose a different user to run your Docker container, make sure that you change the volume to reference the right path for the user home folder.

Now, our .gitlab-ci.yml should look cleaner:

image: maven:3.6-jdk-8
build:
  stage: build
  script:
    - mvn package -U -DskipTests

Multi Stage Pipeline with Maven on Gitlab

We also can provide more steps for our build process. Like, we can include one new stage to check our code style (Lint) before we build the application. So we can improve the code quality.

image: maven:3.6-jdk-8
stages:
  - lint
  - build
    
Lint:
  stage: lint
  script:
    - java -jar checkstyle.jar -c checkstyle-rules.xml src/main/java/com/bitslovers
build:
  stage: build
  script:
    - mvn package -U -DskipTests

Conclusion

You have learned how to configure and use the CI/CD from Gitlab to build a Java application using Maven as package management. Also, now you can properly use the alternatives to use the right cache approach to speed up your pipeline.

Remember, if you have more than one Runner, and you need to pick up the right one to build your Java application, use Gitlab Tags to select the right Runner.

Also, I really recommend the Gitlab Documentation and official Maven Documentation.

Visit Studycrumb

Check also other articles related to Gitlab:

How to use the Gitlab CI Variables

Pipeline to build Docker in Docker on Gitlab.

How to execute Cloud Formation on Gitlab.

How to execute Terraform on Gitlab.

How delete a project on Gitlab.