Sonarqube and Docker, let’s see how to use both technologies to make the best of Quality Control of the release life cycle from your application without spending a tremendous amount of time installing and configuring on the server.
But first, let’s talk about what Sonarqube is and what it does.
Sonarqube Meaning
Sonarqube definition By Wikipedia
SonarQube is an open-source platform developed by SonarSource for continuous inspection of code quality to perform automatic reviews with static code analysis to detect bugs, code smells, and security vulnerabilities on 20+ programming languages.
Why Sonarqube is used?
Nowadays, developers are struggling with challenging deadlines to produce the required functionality for the customer. Developers need to compromise with the code quality, possible bugs, code duplications, and lousy complexity distribution.
Additionally, they tend to leave unused variables, and functions, forget to close a connection, etc. In this situation, the code would work in the desired way.
SonarQube reduces the risk of software development in a concise amount of time. With analysis, automation identifies bugs in the code and warns developers to fix them before releasing them for production. SonarQube also highlights the complicated areas of code that are less covered by unit tests.
Which versions of Sonarquebe are available? Is sonarqube free?
There are four different versions:
1 – Community Edition (Free and Open Source)
2 – Developer (Paid)
3 – Enterprise (Paid)
4 – Data Center (Paid)
For Pricing, check here.
The Community Edition (CE) should provide enough features to get started. However, there are 2 limitations that it’s worth being aware of: First, there is a limitation of how Sonarqube will analyze many code lines for your whole Sonarqube. Second, to analyze the C++ and C# codes, you must buy at least the Developer Edition.
Also, the Community Edition supports analysis on your Javascript project. If you would like to learn how to use Sonarqube to analyze Javascript code check our post.
Sonarqube Source
The source code from Sonarqube is opened and could be modified to attempt your necessity. You can find it on GitHub.
How to install the Sonarqube?
The Sonarqube has three components:
- The SonarQube server runs the following processes:
- The web server provides a user interface.
- Search server based on Elasticsearch.
- The compute engine is in charge of processing code analysis reports and saving them in the SonarQube database.
- The database to store the following:
- Metrics and issues for code quality and security generated during code scans.
- The SonarQube instance configuration.
- One or more scanners running on your build or continuous integration servers to analyze projects.
The easier way to run Sonarqube is using Docker container with Docker Compose using Linux, defining two components on the Yaml file. This method also enables you to run locally without any extra configuration. Furthermore, no database is required to run on the Docker version from Sonarqube because it already has an embedded database. However, it is not an excellent approach to run an embedded database in production because it will limit Sonarqube performance and flexibility to update it.
However, there are two more ways to install the Sonarqube. Let’s list all of them:
1 – on a Windows machine, the Sonarqube can run as a service.
2 – on Kubernetes Cluster.
Sonarqube docker image
You can pull the latest image from DockerHub. To do that, you can perform:
docker pull sonarqube:8.9.0-community
If you don’t care about which version you use and always would like to use the latest version, you can execute:
docker pull sonarqube:latest
Note: If you have a Pipeline process that create a server and install the Sonarqube using ansible for example (or any other automation tool), it's recommeded to specify the a version that you already tested and validated. In other words avoid use "docker pull sonarqube:latest" on production scripts.
Or if you would like to create your own image, for some reason and not use the official one, follow the Sonarqube Dockerfile:
Sonarqube Dockerfile
FROM openjdk:11-jre-slim
RUN apt-get update
&& apt-get install -y curl gnupg2 unzip
&& rm -rf /var/lib/apt/lists/*
ENV SONAR_VERSION=7.9.6
SONARQUBE_HOME=/opt/sonarqube
SONARQUBE_JDBC_USERNAME=sonar
SONARQUBE_JDBC_PASSWORD=sonar
SONARQUBE_JDBC_URL=""
# Http port
EXPOSE 9000
RUN groupadd -r sonarqube && useradd -r -g sonarqube sonarqube
# pub 2048R/D26468DE 2015-05-25
# Key fingerprint = F118 2E81 C792 9289 21DB CAB4 CFCA 4A29 D264 68DE
# uid sonarsource_deployer (Sonarsource Deployer) <[email protected]>
# sub 2048R/06855C1D 2015-05-25
RUN for server in $(shuf -e ha.pool.sks-keyservers.net
hkp://p80.pool.sks-keyservers.net:80
keyserver.ubuntu.com
hkp://keyserver.ubuntu.com:80
pgp.mit.edu) ; do
gpg --batch --keyserver "$server" --recv-keys F1182E81C792928921DBCAB4CFCA4A29D26468DE && break || : ;
done
RUN set -x
&& cd /opt
&& curl -o sonarqube.zip -fSL https://binaries.sonarsource.com/Distribution/sonarqube/sonarqube-$SONAR_VERSION.zip
&& curl -o sonarqube.zip.asc -fSL https://binaries.sonarsource.com/Distribution/sonarqube/sonarqube-$SONAR_VERSION.zip.asc
&& gpg --batch --verify sonarqube.zip.asc sonarqube.zip
&& unzip -q sonarqube.zip
&& mv sonarqube-$SONAR_VERSION sonarqube
&& chown -R sonarqube:sonarqube sonarqube
&& rm sonarqube.zip*
&& rm -rf $SONARQUBE_HOME/bin/*
VOLUME "$SONARQUBE_HOME/data"
WORKDIR $SONARQUBE_HOME
COPY run.sh $SONARQUBE_HOME/bin/
USER sonarqube
ENTRYPOINT ["./bin/run.sh"]
Sonarqube docker compose example:
This example uses the Sonarqube embedded database:
version: "3"
services:
sonarqube:
image: sonarqube:8-community
ulimits:
nofile:
soft: "65536"
hard: "65536"
volumes:
- /tmp/opt/sonarqube/conf:/opt/sonarqube/conf \
- /tmp/opt/sonarqube/logs:/opt/sonarqube/data \
- /tmp/opt/sonarqube/data:/opt/sonarqube/logs \
- /tmp/opt/sonarqube/extensions:/opt/sonarqube/extensions \
ports:
- "9000:9000"This sonarqube docker-compose example specifies a limit for file descriptions of 65536, which is a requirement to run Sonarqube because it uses an embedded Elasticsearch.
If you have this error message:
max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
You can fix that on Linux using the commands below, which will help us to change the limits in your host:
sudo su sysctl -w vm.max_map_count=262144 sysctl -w fs.file-max=65536 ulimit -n 65536 ulimit -u 65536
Sonarqube Docker External Database
version: "3"
services:
sonarqube:
image: sonarqube:8-community
depends_on:
- db
environment:
SONAR_JDBC_URL: jdbc:postgresql://db:5432/sonar
SONAR_JDBC_USERNAME: sonar
SONAR_JDBC_PASSWORD: sonar
ulimits:
nofile:
soft: "65536"
hard: "65536"
volumes:
- /tmp/opt/sonarqube/conf:/opt/sonarqube/conf \
- /tmp/opt/sonarqube/logs:/opt/sonarqube/data \
- /tmp/opt/sonarqube/data:/opt/sonarqube/logs \
- /tmp/opt/sonarqube/extensions:/opt/sonarqube/extensions \
ports:
- "9000:9000"
db:
image: postgres:12
environment:
POSTGRES_USER: sonar
POSTGRES_PASSWORD: sonar
PGDATA: /var/lib/postgresql/data/pgdata
volumes:
- /custom/mount:/var/lib/postgresql/dataYou noticed (from the docker-compose example for Sonarqube) that it is possible to run the Sonarqube using an external database (it’s a best practice). But first, you need to specify the three environment variables:
SONAR_JDBC_URL
SONAR_JDBC_USERNAME
SONAR_JDBC_PASSWORDThis configuration was also tested using Postgres 9.12 (RDS) from AWS, which works pretty well.
Sonarqube MySQL Setup
The new version of Sonarque doesn’t support the MySQL database. However, see below the database currently supported:
How to run the sonarqube docker image?
You can create a run.sh, file, and quickly start the Sonarqube using it. This approach is used when you don’t have a reason to use the docker-compose or need an external database.
mkdir -p ./opt/sonarqube/conf mkdir -p ./opt/sonarqube/data mkdir -p ./opt/sonarqube/logs mkdir -p ./opt/sonarqube/extensions mkdir -p ./opt/postgres export SONARQUBE_DIR=$(pwd)/opt export POSTGRES_DIR=$(pwd)/postgres docker run --detach -p 9000:9000 \ --name sonarqube \ --rm \ --stop-timeout 3600 \ -v $SONARQUBE_DIR/conf:/opt/sonarqube/conf \ -v $SONARQUBE_DIR/logs:/opt/sonarqube/data \ -v $SONARQUBE_DIR/data:/opt/sonarqube/logs \ -v $SONARQUBE_DIR/extensions:/opt/sonarqube/extensions \ sonarqube:8.7.1-community
Basically, we can execute “docker run sonarqube” with some parameters like the example above. As you saw, there is no secret. The Good approach to specify the name of the container is that if you would like to restart or stop the Sonarqube, you can run:
docker stop sonarqube docker restart sonarqube
How to Enable HTTPS on Sonarqube?
Unfortunately, the way to enable HTTPS for Sonarqube is by creating an HTTPS reverse proxy. You can use Ngnix or Apache2. However, it is not hard to do it. But, if your goal is only to test or run locally only, keep this step is only required for a production server.
I have created a docker container using apache2. You can check out the code on:
https://github.com/clebermasters/sonarqube-https-apache2
You need to mount a volume to provide the certificates, like -v /root/certs:/etc/apache2/certs.
# Build the Container cd sonarqube-https-apache2 docker build -t sonarqube-bitslovers-http-proxy:v1 . # Run the container docker run -d -v /root/certs:/etc/apache2/certs --rm -it sonarqube-bitslovers-http-proxy:v1
Inside the /root/certs folder, should have 3 files:
server.crt
server.pem
chain.pemWhen you start the container, the proxy will forward the request from default Sonarqube port 9000 to 80 (HTTP) and 443 (HTTPS).
If you don’t have the certificate files, you can quickly issue them for free using Let’s Encrypt. You can follow these instructions.
Or if you will install Sonarqube on a cloud server like AWS, if you have a Load Balance, you will not need to worry about the certificate; you can handle it on AWS.
How to configure Sonarqube?
The Sonarqube only requires one configuration to get started to scan your first project. You should have a token (See below how to do it), and three essential items would be good to do it:
First, define the user and groups and the permission for each. Second, establish a new Quality Gate for your project or create a new global one if the default does not fit your requirements. If the project doesn’t explicitly has been assigned to a Quality Gate, Sonarqube will use the Global one.
Also, don’t forget to change the default password:
The default user with Administer System permission is admin, and the password: admin
How to analyze a project?
First, for authentication, we need a token to call the Sonarqube from our pipeline. Here, we will use Maven as our tool to build our application.
Suppose you see any issue related to “maven sonar not authorized” or “Not authorized. Analyzing this project requires authentication. Please provide a user token in sonar.login or other credentials in sonar.login and sonar.password.” So, it would help if you created a token.
How to create a Token?
You can create new tokens at User > My Account > Security. The form at the bottom of the page enables you to create new tokens. Once you click the Generate button, you will observe the token value. Copy it immediately; once you dismiss the notification, you will not be able to recover it.
Copy your token to a safe place because it will not be displayed again on the screen above.
Step to analyze your first project:
Create a Project on Sonarqube:
A project is generated in SonarQube automatically on its first analysis. But, if you require to set some configuration on your project before its first analysis, you have the possibility of provisioning it via Administration options or the + menu item, which is visible to users with project creation rights.
To integrate the build process with the Sonarqube analysis, we need to create a maven profile and specify its goal. There are two approaches to doing it. First, you can change each project on the pom.xml file by adding the maven profile. Second, you can change the global maven configuration file, usually located on ~/.m2/settings.xml. This last approach looks like a perfect one; if you have too many projects, you don’t want to go through all of them to change the pom.xml for each one.
For the maven project, add this configuration to your settings.xml file under your ~/.m2/
<settings>
<pluginGroups>
<pluginGroup>org.sonarsource.scanner.maven</pluginGroup>
</pluginGroups>
<profiles>
<profile>
<id>sonar</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<properties>
<!-- Optional URL to server. Default value is http://www.bitslovers.com:9000 -->
<sonar.host.url>
http://myserver:9000
</sonar.host.url>
<sonar.login>
93744265cebbe989946c0250b96c09fa8b20732b
</sonar.login>
<sonar.scm.provider>
git
</sonar.scm.provider>
</properties>
</profile>
</profiles>
</settings>
<pluginGroups>
<pluginGroup>org.sonarsource.scanner.maven</pluginGroup>
</pluginGroups>
Remember to change myserver:9000 to your server and port. Also, notice that we have added sonar.login with our token, so we don’t need to specify it every time you call the Sonarqube. Do not forget to replace it with yours too.
Another important attribute is sonar.scm.provider. We know that most projects use Git today, but you should change this value if you do not.
Examining a Maven project consists of running a Maven goal sonar:sonar from the directory that contains the main project pom.xml. Next, you need to pass an authentication token applying the sonar.login property in your command line.
Sonarqube maven-plugin
The approach above adds the maven-plugin (lines 21-23) to use Sonarqube globally. In other words, you don’t need to add the maven-plugin for Sonarqube for each project if you have more than one. Important note that the plugin doesn’t run the Sonarqube automatically. You have to specify the maven goal sonar:sonar to trigger the Sonaqube on your install or package phase.
Run Sonarqube with Maven without Global Setting (without editing the settings.xml)
We are not building the application and requesting maven to send the report to our Sonarqube server. Instead, building application is required to go through the analysis phase.
mvn sonar:sonar -Dsonar.projectKey=Devops-API -Dsonar.host.url=http://www.bitslovers.com -Dsonar.scm.provider=git -Dsonar.login=93744265cebbe989946c0250b96c09fa8b20732b
Otherwise, if building the application should also trigger the analysis on our server:
mvn sonar:sonar package -U -Dsonar.projectKey=Devops-API -Dsonar.host.url=http://www.bitslovers.com -Dsonar.scm.provider=git -Dsonar.login=93744265cebbe989946c0250b96c09fa8b20732b
If the settings.xml is configured, running the analysis is pretty simple:
mvn clean verify sonar:sonar -Dsonar.login=myAuthenticationToken
Which configuration should I change to fail a project if I do not reach the rules?
Quality Gates
Quality gates are acceptance rules reports that a role can use throughout any project. Indeed, managers of smaller, agile projects might declare that this requires too endless documentation. But, the decent idea about quality gates is that the design is entirely customizable. It can describe a set of predefined quality guidelines that a software development project must match to proceed from one stage of its lifecycle to the next.
Sonarqube comes with a predefined set of Quality Gates. But you can define your own. These roles are the guidelines used by Sonarqube to report if your project passed on the analysis.
The analysis could be integrated and combined with a pipeline-building process, generally one stage of the application’s lifecycle, before release. So it means that when you trigger the analysis into sonarqube after committing any changes in your project will tell two possibilities:
First, your project meets with all Quality Gates, and the project will follow the next stage of the lifecycle.
Second, the reports detected a deviation in at least one role from Quality Gates.
For best practices, the pipeline should fail and prevent releasing a new version of one application if the current changes for a specific project do not meet with requirements of Quality Gates. However, by default, the sonarqube and maven will not fail the build pipeline if the project does not meet the Quality Gates roles. We must provide the parameter below when we call the maven to build our application; the maven will wait for the analysis to finish the process, and the result as passed or failed. Finally, the pipeline will deploy the application to your repository if it’s the case.
-Dsonar.qualitygate.wait=true -Dsonar.qualitygate.timeout=3000
What are the possibles criteria for Quality Gates that works with Sonarqube?
Follow below the list (not all) of conditions/roles on Sonarqube:
Added Technical Debt
Blocker Issues
Bugs
Code Smells
Cognitive Complexity
Complexity / Class
Complexity / File
Complexity / Function
Complexity in Classes
Complexity in Functions
Condition Coverage
Condition Coverage on New Code
Critical Issues
Duplicated Blocks
Duplicated Blocks on New Code
Duplicated Files
Duplicated Lines
Major Issues
Minor Issues
New Blocker Issues
New Bugs
New Code Smells
New Critical Issues
New Security Hotspots
New Vulnerabilities
Public Undocumented API
Reliability Rating
Security Rating on New Code
Skipped Unit Tests
Technical Debt
Unit Test Errors
Unit Test Failures
Unit Test Success (%)
Vulnerabilities
More detail about Quality Gates.
Jenkins with Sonarqube
Running a SonarQube scan from a pipeline on your local computer is fine. However, a robust solution requires incorporating SonarQube as part of the continuous integration process. For example, if you combine SonarQube analysis into a Jenkins pipeline, you can guarantee that the pipeline won’t proceed to a further stage to release if the quality gate fails. In addition, nobody desires to release lousy code into production.
So, we can utilize the SonarQube Scanner plugin for Jenkins. It incorporates two peculiarities that we’re going to talk about today:
SonarQube server configuration – the plugin allows you to place your SonarQube server location and credentials. This data is then used in a SonarQube analysis pipeline to transmit code analysis reports to that SonarQube server.
SonarQube Quality Gate webhook – when a code analysis report is offered to SonarQube, unluckily, it doesn’t respond synchronously with the result of whether the information passed the quality gate or not. So to do this, The Plugin needs to configure a webhook call in SonarQube to ask back into Jenkins to concede our pipeline to continue (or fail). The SonarQube Scanner Jenkins plugin makes this webhook possible.
- Let’s list a complete analysis of the interaction between Jenkins and SonarQube:
- Jenkins pipeline began.
- The SonarQube scanner is run upon a code project, and the analysis report is transferred to the SonarQube server.
- SonarQube ends the investigation and reviews the project meets the configured Quality Gate.
- SonarQube transmits a pass or defeats the result back to the Jenkins webhook exposed by the plugin.
Sonar Maven Plugin
Sonarqube Maven Jenkins Plugin
Also, be aware that you don’t need to install a Sonarqube plugin for Jenkins to integrate it. Instead, you can leverage the Maven plugin to configure your project or the global configuration (settings.xml).
Sonarqube and Gitlab
Integrate the Sonarqube with Gitlab. It’s very straightforward and also not require any additional plugins. However, you need to configure your pom.xml or settings.xml (the best approach is to set this file on your Runner) to call the Sonarqube. I will cover more details about this integration in my next post.
Conclusion
Sonarqube with Docker and Maven it’s the perfect combination for your local workspace daily. This Sonarqube tutorial is only a few parts of what you can do with those technologies, there is so much more content to cover, but I believe you have enough information to get started with this article. Also, we are preparing new content to cover more scenarios around this topic, and please stay tuned.
In addition, we covered the process of analyzing Java code using Maven as a build tool, but we also can utilize the Gradle instead of Maven.
To analyze other languages, like C#, C++, or Javascript projects, you have to use the Sonarqube scanner, a binary that doesn’t require any installation. To learn more, check here.
Please, join our telegram channel or on Twitter and receive our last updates!
You have two pluginGroups declared in your settings.xml. The pluginGroups needs to be inside settings.xml