Skip to content

Measure Code Coverage

This guide adds code coverage, so Run with Coverage shows which lines and branches your tests execute. It takes a coverage tool and a small mixin that enables one of Ceedling’s coverage plugins, Gcov or Bullseye. At the end, the Test Coverage panel lists each source file’s coverage, and the editor marks each line’s hits.

What you need

  • A project the Testing view already lists. See Start Testing in Your Project.
  • A coverage tool where Ceedling runs, from one of the plugins’ choices. Code Coverage compares them.
    • gcov, free with GCC, with a report tool, either gcovr or ReportGenerator. gcovr is the usual choice. ReportGenerator, a .NET tool, is common on Windows. Ceedling’s Gcov plugin builds its coverage tests with GCC, whatever compiler the project’s other tests use.
    • Bullseye, a commercial, licensed tool, with Ceedling 1.2.0 or later.

Steps

  1. Install the coverage tool, in the shell Ceedling runs in.
  2. Save a coverage mixin as coverage.yml in the project’s directory, beside project.yml.
  3. Add the mixin to the folder’s .vscode/settings.json.
  4. Run with coverage. Choose Run with Coverage for a test, a file, or the whole project in the Testing view. The Test Coverage panel opens with the results.

Install gcovr with Python’s pip.

pip install gcovr

The mixin enables the Gcov plugin, with gcovr writing the Cobertura report the extension reads.

:plugins:
  :enabled:
    - gcov
:gcov:
  :utilities:
    - gcovr
  :reports:
    - Cobertura

Install ReportGenerator as a .NET global tool.

dotnet tool install -g dotnet-reportgenerator-globaltool

The mixin enables the Gcov plugin, with ReportGenerator writing the Cobertura report the extension reads.

:plugins:
  :enabled:
    - gcov
:gcov:
  :utilities:
    - ReportGenerator
  :reports:
    - Cobertura

Install Bullseye from its download page, with your license key, and put its bin directory on the shell’s PATH. Here the key comes from an environment variable, which keeps it out of the shell’s history. Ceedling’s Bullseye licensing page explains each kind of license.

./install --prefix ~/bullseye --key "$BULLSEYE_LICENSE_KEY" --search "$PATH"
export PATH=~/bullseye/bin:$PATH

The mixin enables the Bullseye plugin, with its Cobertura report, which the extension reads.

:plugins:
  :enabled:
    - bullseye
:bullseye:
  :xml_report: :cobertura

The setting applies the mixin to every project in the folder that names no mixins of its own. A mixin’s path is relative to the project’s directory.

{
    "ceedlingExplorer.mixins": ["coverage.yml"]
}

A mixin keeps the plugin out of project.yml, so builds outside VS Code stay as they were. Every run has the plugin available, and only coverage runs use it. The same settings can go in project.yml instead.

Variations

Each variation below changes one thing about the basic setup.

Sources above the project

gcovr reports only sources beneath its report root, which is the project’s directory by default. A project whose sources sit above it, e.g. in a sibling ../src, needs a wider root. ReportGenerator includes every source directory in :paths: already, and Bullseye every source its builds compile.

:gcov:
  :gcovr:
    :report_root: ".."

Coverage across several runs

Each coverage run shows only what it executed, since the extension clears earlier runs’ measurements first. To add each run’s coverage to the last, as Ceedling itself does, turn the reset off. It applies to either plugin.

{
    "ceedlingExplorer.coverageResetsCounters": false
}

The measurements then accumulate until ceedling clean or ceedling clobber. The Testing view’s Clean button clears them.

Reports from a separate task

A project whose plugin sets :report_task: to TRUE writes reports only from that plugin’s report: task. The extension adds that task to each coverage run by itself, so no other setting is needed.

With gcov, reports then come from report:gcov.

:gcov:
  :report_task: TRUE

With Bullseye, they come from report:bullseye.

:bullseye:
  :report_task: TRUE

Both plugins

A project may enable both plugins, each in its own mixin or together in one. The Testing view then offers a profile for each, Coverage (gcov) and Coverage (Bullseye), and each measures through its own plugin. Code Coverage explains how they share a project.

When no coverage appears

A coverage run that shows nothing usually says why in a notification. The usual causes are a coverage tool missing from the shell’s PATH, a report other than Cobertura, or, for Bullseye, a missing license. Code Coverage lists each reason and its fix.