Code Coverage API plugin: 1.0 Release

Published on 2018-08-17 by Shenyu Zheng

I am happy to announce availability of Code Coverage API. These plugins have been recently released as 1.0, and they are now available in the Jenkins Update Center. In this blogpost I will introduce the features and project structure of Code Coverage API plugin.

My name is Shenyu Zheng, and I am an undergraduate student in Computer Science and Technology at Henan University from China.

Overview

Code Coverage API plugin is one of GSoC 2018 Jenkins projects.

There are a lot of plugins which currently implement code coverage; however, they all use similar config, charts, and content. So it would be much better if we could have an API plugin which does the most repeated work for those plugins and offers a unified API which can be consumed by other plugins and external tools.

My mentors are Steven Christou, Supun Wanniarachchi, Jeff Pearce and Oleg Nenashev.

Supported Coverage Formats

Features

  • Modernized coverage chart

  • Coverage trend

  • Source code navigation

  • Parallel pipeline support

  • Reports combining

  • REST API

  • Failed conditions and flexible threshold setting

  • Other small features

Modernized Coverage Chart

In the summary chart we can see the coverage summary of current coverage metric. summary chart

In the child summary chart, we can see the coverage summary of each child, also, we can use the range handler to filter item we want to see to reduce the chart size. If we want to see coverage details of the child, we can click the child name to see more information. child summary chart

Coverage Trend

We also support coverage trend to show coverage metrics changing between builds. trend chart

Source Code Navigation

You can enable source code navigation by specifying Source File Storing Level to save last build source files (enable source files navigation in current and last build) or save all build source files (enable source files navigation in all builds). source files config

You can see source file with coverage information on File level coverage page. source files result

Parallel Pipeline Support

We support parallel pipeline. You can call the Code Coverage API plugin in different branches like this:

node {
    parallel firstBranch: {
        publishCoverage adapters: [jacocoAdapter('target/site/jacoco/jacoco.xml')]
}, secondBranch: {
        publishCoverage adapters: [jacocoAdapter('jacoco.xml')]
    }
}

Reports Combining

You can add tag on publishCoverage and Code Coverage API plugin will combine reports have same tag

node {
    parallel firstBranch: {
        publishCoverage adapters: [jacocoAdapter('target/site/jacoco/jacoco.xml')], tag: ‘t’
}, secondBranch: {
        publishCoverage adapters: [jacocoAdapter('jacoco.xml')], tag: ‘t’
    }
}

REST API

We provide a REST API to retrieve coverage data:

  • Coverage result: …​/{buildNumber}/coverage/…​/result/api/\{json|xml\}

  • Trend result: …​/{buildNumber}/coverage/…​/trend/api/\{json|xml\}

  • Coverage result of last build: …​/{buildNumber}/coverage/…​/last/result/api/\{json|xml\}

  • Trend result of last build: …​/{buildNumber}/coverage/…​/last/trend/api/\{json|xml\}

Failed Conditions and Flexible Threshold Setting

You can set different failed conditions and threholds to control build result. thresholds config

If the thresholds satisfy the failed conditions, it will fail the build. thresholds result

Other Small Features

We also have other small features like auto detecting reports, coverage filters, etc. You can find more information about these features in the plugin documentation.

Architecture

This API plugin will mainly do these things:

  • Find coverage reports according to the user’s config.

  • Use adapters to convert reports into the our standard format.

  • Parse standard format reports, and aggregate them.

  • Show parsed result in a chart.

So, we can implement code coverage publishing by simply writing an adapter, and such adapter only needs to do one thing - convert a coverage report into the standard format. The implementation is based on extension points, so new adapters can be created in separate plugins. In order to simplify conversion for XML reports, there is also an abstraction layer which allows creating XSLT-based converters.

The below diagram show the architecture of Code Coverage API plugin

architecture

Implementing a New Coverage Plugin

We can implement a coverage plugin by implementing CoverageReportAdapter extension point. For example, by using the provided abstract layer, we can implement JaCoCo simple like this:

public final class JacocoReportAdapter extends JavaXMLCoverageReportAdapter {

    @DataBoundConstructor
    public JacocoReportAdapter(String path) {
        super(path);
    }

    /**
     * {@inheritDoc}
     */
    @Override
    public String getXSL() {
        return "jacoco-to-standard.xsl";
    }

    /**
     * {@inheritDoc}
     */
    @Override
    public String getXSD() {
        return null;
    }

    @Symbol("jacoco")
    @Extension
    public static final class JacocoReportAdapterDescriptor extends JavaCoverageReportAdapterDescriptor {

        public JacocoReportAdapterDescriptor() {
            super(JacocoReportAdapter.class);
        }

        @NonNull
        @Override
        public String getDisplayName() {
            return Messages.JacocoReportAdapter_displayName();
        }
    }
}

All we need is to extend an abstract layer for XML-based Java report and provide an XSL file to convert the report to our standard format. There are also other extension points which are under development.

If you want implement a new coverage format that we did not provide abstract layer, you need to register `CoverageElement`s and implement an simple parser. See llvm-cov Plugin to get more details.

Future Tasks

Phase 3 Presentation Slides

Phase 3 Presentation Video

About the Author
Shenyu Zheng

Shenyu comes from China. He is a third year student now, and his major is Computer Science and technology. He has participated in GSoC 2018 for Code Coverage API Plugin