We launch inspections IntelliJ IDEA on Jenkins

We launch inspections IntelliJ IDEA on Jenkins


IntelliJ IDEA today has the most advanced static Java code analyzer, which in its capabilities has left behind such "veterans" as Checkstyle and Spotbugs . Her numerous “inspections” check the code in various aspects, from the coding style to characteristic bugs.


However, while the analysis results are displayed only in the developer's local IDE interface, they have little use for the development process. Static analysis needs to be performed as the first step of the assembly line, its results should determine quality gates, and the assembly should be fayl if quality gates not passed. It is known that TeamCity CI is integrated with IDEA. But even if you are not using TeamCity, you may well try to run IDEA inspections on any other CI server. I propose to see how this can be done using the IDEA Community Edition, Jenkins and Warnings NG plugin.


Step 1. Run the analysis in the container and get a report


At first, the idea to launch an IDE (desktop application!) within a CI system that does not have a graphical interface may seem dubious and very troublesome. Fortunately, IDEA developers have provided the ability to run code formatting and inspections from the command line. Moreover, to run IDEA in this mode does not require a graphics subsystem and these tasks can be performed on servers with a text shell.


Inspections are launched using the bin/inspect.sh script from the IDEA installation directory. The following parameters are required:


  • full path to the project (relative is not supported),
  • path to the .xml file with inspection settings (usually located inside the project in .idea/inspectionProfiles/Project_Default.xml),
  • The full path to the folder where the .xml files with the analysis results reports will be stored.

Also, it is expected that


  • The IDE will configure the path to the Java SDK, otherwise the analysis will not work. These settings are contained in the jdk.table.xml configuration file in the global configuration folder IDEA. The IDEA global configuration itself is in the user's home directory by default, but this location can be explicitly set in the file idea.properties .
  • The analyzed project should be a valid IDEA project, for which you will have to commit some files to version control, which are:
    • .idea/inspectionProfiles/Project_Default.xml - analyzer settings, they will obviously be used when starting inspections in a container,
    • .idea/modules.xml - otherwise we get the error 'This project contains no modules',
    • .idea/misc.xml - otherwise we get the error 'The JDK is not configured properly for this project',
    • *. iml-files - otherwise we will get an error about not configured JDK in the module.

Although these files are usually included in .gitignore , they do not contain any information specific to the developer-specific environment — unlike, for example, the workspace.xml file, where such the information is just contained, and therefore it is not necessary to commit it.


Itself suggests a way to pack the JDK along with IDEA Community Edition into a container in the form ready for setting up on the analyzed projects.Choose a suitable base container, and this is how we get the Dockerfile:


Dockerfile
  FROM openkbs/ubuntu-bionic-jdk-mvn  -py3

 ARG INTELLIJ_VERSION = "ideaIC-2019.1.1"

 ARG INTELLIJ_IDE_TAR = $ {INTELLIJ_VERSION} .tar.gz

 ENV IDEA_PROJECT_DIR = "/var/project"

 WORKDIR/opt

 COPY jdk.table.xml/etc/idea/config/options/

 RUN wget https://download-cf.jetbrains.com/idea/${INTELLIJ_IDE_TAR} & amp; & amp;  \
  tar xzf $ {INTELLIJ_IDE_TAR} & amp; & amp;  \
  tar tzf $ {INTELLIJ_IDE_TAR} |  head -1 |  sed -e 's/\/.*//' |  xargs -I {} ln -s {} idea & amp; & amp;  \
  rm $ {INTELLIJ_IDE_TAR} & amp; & amp;  \
  echo idea.config.path =/etc/idea/config & gt; & gt;  idea/bin/idea.properties & amp; & amp;  \
  chmod -R 777/etc/idea

 CMD idea/bin/inspect.sh $ {IDEA_PROJECT_DIR} $ {IDEA_PROJECT_DIR}/. Idea/inspectionProfiles/Project_Default.xml $ {IDEA_PROJECT_DIR}/target/idea_inspections -v2  

Using the idea.config.path option, we forced IDEA to search for its global configuration in the /etc/idea folder, since the user's home folder in the conditions of work CI is an indefinite thing and often completely absent.


This is the jdk.table.xml file that is copied to the container and contains the paths to the OpenJDK installed inside the container (a similar file from your own directory with the IDEA settings can be used as the basis):


jdk.table.xml
  & lt; application & gt;
  & lt; component name = "ProjectJdkTable" & gt;
  & lt; jdk version = "2" & gt;
  & lt; name value = "1.8"/& gt;
  & lt; type value = "JavaSDK"/& gt;
  & lt; version value = "1.8"/& gt;
  & lt; homePath value = "/usr/java"/& gt;
  & lt; roots & gt;
  & lt; annotationsPath & gt;
  & lt; root type = "composite" & gt;
  & lt; root url = "jar://$APPLICATION_HOME_DIR$/lib/jdkAnnotations.jar!/" type = "simple"/& gt;
  & lt;/root & gt;
  & lt;/annotationsPath & gt;
  & lt; classPath & gt;
  & lt; root type = "composite" & gt;
  & lt; root url = "jar:///usr/java/jre/lib/charsets.jar!/" type = "simple"/& gt;
  & lt; root url = "jar:///usr/java/jre/lib/deploy.jar!/" type = "simple"//gt;
  & lt; root url = "jar:///usr/java/jre/lib/ext/access-bridge-64.jar!/" type = "simple"/& gt;
  & lt; root url = "jar:///usr/java/jre/lib/ext/cldrdata.jar!/" type = "simple"/& gt;
  & lt; root url = "jar:///usr/java/jre/lib/ext/dnsns.jar!/" type = "simple"/& gt;
  & lt; root url = "jar:///usr/java/jre/lib/ext/jaccess.jar!/" type = "simple"/& gt;
  & lt; root url = "jar:///usr/java/jre/lib/ext/jfxrt.jar!/" type = "simple"/& gt;
  & lt; root url = "jar:///usr/java/jre/lib/ext/localedata.jar!/" type = "simple"/& gt;
  & lt; root url = "jar:///usr/java/jre/lib/ext/nashorn.jar!/" type = "simple"/& gt;
  & lt; root url = "jar:///usr/java/jre/lib/ext/sunec.jar!/" type = "simple"/& gt;
  & lt; root url = "jar:///usr/java/jre/lib/ext/sunjce_provider.jar!/" type = "simple"/& gt;
  & lt; root url = "jar:///usr/java/jre/lib/ext/sunmscapi.jar!/" type = "simple"/& gt;
  & lt; root url = "jar:///usr/java/jre/lib/ext/sunpkcs11.jar!/" type = "simple"/& gt;
  & lt; root url = "jar:///usr/java/jre/lib/ext/zipfs.jar!/" type = "simple"/& gt;
  & lt; root url = "jar:///usr/java/jre/lib/javaws.jar!/" type = "simple"/& gt;
  & lt; root url = "jar:///usr/java/jre/lib/jce.jar!/" type = "simple"/& gt;
  & lt; root url = "jar:///usr/java/jre/lib/jfr.jar!/" type = "simple"/& gt;
  & lt; root url = "jar:///usr/java/jre/lib/jfxswt.jar!/" type = "simple"/& gt;
  & lt; root url = "jar:///usr/java/jre/lib/jsse.jar!/" type = "simple"//gt;
  & lt; root url = "jar:///usr/java/jre/lib/management-agent.jar!/" type = "simple"/& gt;
  & lt; root url = "jar:///usr/java/jre/lib/plugin.jar!/" type = "simple"/& gt;
  & lt; root url = "jar:///usr/java/jre/lib/resources.jar!/" type = "simple"/& gt;
  & lt; root url = "jar:///usr/java/jre/lib/rt.jar!/" type = "simple"/& gt;
  & lt;/root & gt;
  & lt;/classPath & gt;
  & lt;/roots & gt;
  & lt; additional/& gt;
  & lt;/jdk & gt;
  & lt;/component & gt;
 & lt;/application & gt;  

Ready-made image is available on the Docker Hub .


Before moving on, let's check the launch of the IDEA analyzer in the container:


  docker run --rm -v & lt; path/to/your/project & gt;:/var/project inponomarev/intellij-idea-analyzer  

The analysis should work successfully, and numerous .xml-files with analyzer reports should appear in the target/idea_inspections subfolder.


Now there is no longer any doubt that the IDEA analyzer can be run offline in any CI environment, and we move on to the second step.


Step 2. Display and analyze the report


Get the report in the form of .xml-files - half the battle, now you need to make it human-readable. And also its results should be used in quality gates - the logic of determining whether the accepted change passes or does not pass according to quality criteria.


Jenkins Warnings NG Plugin will help us in this, released in January 2019. With its appearance, many separate plugins for working with static analysis results in Jenkins (CheckStyle, FindBugs, PMD, etc.) are now marked obsolete.


The plugin consists of two parts:


  • multiple parser message collectors ( the complete list includes everything well-known analyzers from AcuCobol to ZPT Lint),
  • a single report viewer for all of them.

In the list of what Warnings NG can analyze, there are also warnings from the Java compiler and warnings from the Maven execution logs: although they are constantly visible, they are rarely deliberately analyzed. IntelliJ IDEA reports are also included in the list of recognizable formats.


t. K. plugin new, it initially interacts well Jenkins Pipeline. The assembly step with his participation will look like this (we just tell the plugin which report format we recognize and which files should be scanned):


  stage ('Static analysis') {
  sh 'rm -rf target/idea_inspections'
  docker.image ('inponomarev/intellij-idea-analyzer'). inside {
  sh '/opt/idea/bin/inspect.sh $ WORKSPACE $ WORKSPACE/.idea/inspectionProfiles/Project_Default.xml $ WORKSPACE/target/idea_inspections -v2'
  }
  recordIssues (
  tools: [ideaInspection (pattern: 'target/idea_inspections/*. xml')]
  )
 }  

Report interface looks like this:



Conveniently, this interface is universal for all recognized analyzers. It contains an interactive chart of the distribution of finds by categories and a graph of the dynamics of changes in the number of finds. In the grid at the bottom of the page you can perform a quick search. The only thing that did not work correctly for IDEA bakes is the ability to browse the code directly in Jenkins (although for other reports, such as Checkstyle, this plugin can do this beautifully). It looks like this is a bug of IDEA report parser that needs to be fixed.


Among the features of Warnings NG is the ability to aggregate finds from different sources in one report and program Quality Gates, including the “ratchet” by reference assembly. Some Quality Gates programming documentation is available here - however, it is not complete, and you have to look at the source. On the other hand, for complete control over what is happening, the ratchet can be implemented independently (see my previous post on this topic ).


Conclusion


Before you start preparing this material, I decided to search: didn’t anyone write on this topic on Habré already? I found only a 2017 interview with lany , where he says:


As far as I know, there is no integration with Jenkins or maven-plugin [...] In principle, any enthusiast could make friends with IDEA Community Edition and Jenkins, many would benefit from this.

What: after two years we have the Warnings NG Plugin, and finally this friendship has come true!

Source text: We launch inspections IntelliJ IDEA on Jenkins