Fixes + improvements for exlusion processing #663

latk · 2022-08-21T11:28:44Z

This is an (unfortunately large) PR that addresses various interrelated issues with exclusion processing.

Excluded lines are now represented correctly in the coverage data model (compare Generate also a Report of Excluded Coverage #409, Generate also a Report of Excluded Coverage #503)

Previously, there was support for annotating excluded lines, but the coverage parser didn't yet use those features 😬

To test this properly, JSON reference files were added as well.
In the JSON report, the gcovr/excluded and gcovr/noncode fields are usually false. In a change to the JSON format, these fields are now optional when false.
In fact, the “noncode” concept was removed entirely. Previously, there were two representations for noncode status:
- LineCoverage was absent
- LineCoverage was present but with noncode=True
This had previously also caused difficulties for merging coverage data.

Now, noncode lines are removed from the coverage data model. This changes the output of the JSON writer, but doesn't directly affect other writers. Removal of noncode entries is in line with the upstream GCC-gcov format, and should therefore simplify use of Gcovr-JSON for downstream consumers.
I took this opportunity to write detailed documentation for the JSON output formats.
I fixed minor related bugs relating to the treatment of absent lines in the Coveralls and HTML writers. Sometimes, there can be source code lines for which there is no coverage data, or the original source file cannot be loaded. These changes address relevant edge cases that became visible through the removal of noncode LineCoverage entries.
Looking at the exclusion processing (the original motivation for this PR), problems with exclusion markers became visible. Consider the following code snippet:
```
a();
b(); // GCOVR_EXCL_START
c();
...
x();
y(); // GCOVR_EXCL_STOP
z();
```
Previously, x() but not y() would be excluded – markers behaved like half-open Python ranges. This is probably unintuitive for users.

Now, the line with the STOP marker (y()) will also be excluded. This required changes to unit tests, but not to integration tests. This behaviour is now clearly documented.
I used my now-improved understanding of the exclusion processing to disentangle exclusion processing from the gcov-file parser. This greatly simplifies the parser, removing about 40% of the code of that module. Exclusions are now managed in the gcovr.exclusions.* modules. They operate as separate passes over FileCoverage objects, which should also make it easier to understand how individual exclusion features work.

In the future, this separation between parsing and exclusion processing will make it possible to consume other input formats without having to re-implement the exclusions. In particular, this is a prerequisite for supporting upstream GCC-gcov JSON as an input format (see Support GCOV intermediate format #282).

If the size of this PR is causing problems for review, it can be split into three parts: (1) fixes to exclusion processing, (2) removal of noncode concept, and (3) refactoring/extraction of exclusion+noncode processing.

A line can be in one of the following states: * noncode * coverable * excluded * reportable * covered * uncovered Reportable lines are used for calculating coverage statistics.

fixes gcovr#409 fixes gcovr#503

The reference now clearly describes every field, and also covers the "decision" and "function" elements.

The line coverage data needs a `null` entry for lines without data. Previously, null entries were generated to pad until the last known line. Now, the corresponding source file is read to determine the actual number of lines, which works regardless of whether the last line contains coverage data.

Gcovr previously had a concept of "noncode" lines that do not carry meaningful coverage but nevertheless are represented with a LineCoverage object. This was complex and confusing. Now, noncode status is represented simply by the absence of a matching LineCoverage object. This does slightly change the JSON output, but does not affect any other output formats.

Previously, one to few dummy lines were inserted for nonexistent files

Previously, the markers behaved like Python ranges, with the end being exclusive. This doesn't really make sense when considering code like a(); b(); // GCOVR_EXCL_START c(); ... x(); y(); // GCOVR_EXCL_STOP z(); Previously, x() but not y() would be excluded. Now, the range from b() to including y() will be excluded.

This *greatly* simplifies the size of "gcov_parser", reducing it by about 40%. Since exclusions can now be managed independently from the textual gcov format, it now becomes more easily possible to consume other input formats as well.

codecov · 2022-08-21T11:32:41Z

Codecov Report

Base: 95.34% // Head: 95.49% // Increases project coverage by +0.15% 🎉

Coverage data is based on head (cdd95d4) compared to base (7111438).
Patch coverage: 99.54% of modified lines in pull request are covered.

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #663      +/-   ##
==========================================
+ Coverage   95.34%   95.49%   +0.15%     
==========================================
  Files          24       27       +3     
  Lines        3498     3506       +8     
  Branches      615      605      -10     
==========================================
+ Hits         3335     3348      +13     
+ Misses         93       90       -3     
+ Partials       70       68       -2

Flag	Coverage Δ
ubuntu-20.04	`94.40% <99.54%> (+0.15%)`	⬆️
windows-2019	`95.17% <99.54%> (+0.15%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
gcovr/configuration.py	`99.68% <ø> (ø)`
gcovr/merging.py	`82.95% <ø> (-1.26%)`	⬇️
gcovr/tests/test_gcov_parser.py	`100.00% <ø> (ø)`
gcovr/writer/cobertura.py	`100.00% <ø> (ø)`
gcovr/writer/coveralls.py	`93.63% <ø> (+0.30%)`	⬆️
gcovr/writer/html.py	`95.60% <ø> (+0.30%)`	⬆️
gcovr/writer/json.py	`97.05% <ø> (+0.04%)`	⬆️
gcovr/gcov_parser.py	`98.75% <95.65%> (-0.24%)`	⬇️
gcovr/coverage.py	`99.05% <100.00%> (-0.01%)`	⬇️
gcovr/decision_analysis.py	`91.25% <100.00%> (ø)`
... and 8 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

Spacetown · 2022-08-21T20:39:25Z

To review this, I need a computer. I can take a look in 2 weeks

Spacetown

This makes the control flow much clearer. Only some minor questions.

doc/source/output/json.rst

gcovr/coverage.py

gcovr/exclusions/markers.py

gcovr/gcov.py

gcovr/writer/coveralls.py

gcovr/writer/html.py

Spacetown · 2022-09-09T15:47:56Z

@latk While investigating #669 I've foung http://ltp.sourceforge.net/coverage/lcov/geninfo.1.php. Here the stop line is documented as exclusive:

       LCOV_EXCL_LINE
              Lines containing this marker will be excluded.
       LCOV_EXCL_START
              Marks the beginning of an excluded section. The current line  is
              part of this section.
       LCOV_EXCL_STOP
              Marks  the end of an excluded section. The current line not part
              of this section.

I think, the old behavior was compatible to LCOV.

latk · 2022-12-04T21:13:30Z

I think, the old behavior was compatible to LCOV.

I agree. Of course I think that my fixed behaviour makes more sense and is easier to understand. But compatibility with LCOV would be more important to me.

So that aspect of this PR must be reverted/removed before it can be merged.

This was introduced with gcovr#644.

latk

Thank you @Spacetown for updating the PR. Some observations.

gcovr/coverage.py

gcovr/tests/simple1/reference/gcc-5/coverage-details-linkcss.functions.html

latk · 2022-12-05T12:28:38Z

gcovr/tests/simple1/reference/gcc-5/coveralls.json

+        "head": {
+            "author_email": "[email protected]",
+            "author_name": "Spacetown",
+            "committer_email": "[email protected]",
+            "committer_name": "Spacetown",
+            "id": "1dcf374b131442eb79b6c445fa6aef10c4ee3cfa",
+            "message": "Remove outdated documentation about unsupported branch exclusion."
+        },
+        "remotes": [
+            {
+                "name": "latk",
+                "url": "https://github.com/latk/gcovr.git"
+            },
+            {
+                "name": "origin",
+                "url": "https://github.com/Spacetown/gcovr.git"
+            },
+            {
+                "name": "upstream",
+                "url": "https://github.com/gcovr/gcovr.git"
+            }


we should prevent irrelevant personal data like this from being included in the test reference data

There is a regex to remove this. I'll try to find the problem. Maybe it's easier to convert string to JSON, remove keys and convert to string again.

Ok. The scrubbing is working but for the reference data the original file is copied. Shall we create a new file with the scrubbed data?

Spacetown · 2022-12-05T20:38:15Z

@latk I think you can merge the PR. The issue with the scrubbing of private data should be addressed in a separate PR.

Spacetown

LGFM.

Fixes + improvements for exclusion processing

latk added 18 commits August 21, 2022 12:48

Introduce concept of "reportable" lines

2216071

A line can be in one of the following states: * noncode * coverable * excluded * reportable * covered * uncovered Reportable lines are used for calculating coverage statistics.

correctly mark lines as excluded

0a63153

fixes gcovr#409 fixes gcovr#503

update reference data for coverage exclusion markers

9d112ff

add json reference to excl-line test

e3bf178

json: make gcovr/noncode and gcovr/excluded fields optional

14a3161

update test reference files

bf1351e

docs: updated JSON format reference

c64d2d6

The reference now clearly describes every field, and also covers the "decision" and "function" elements.

html writer: fix handling of nonexistent and empty files

26b1dea

Previously, one to few dummy lines were inserted for nonexistent files

bump JSON format version for "gcovr/noncode" removal

a3aae8d

update test reference files

20c1d38

extract logic for exclusions

12a45e3

split exclusions code into separate modules

642d58b

update changelog

595a877

update reference files for Windows

5448325

latk added Type: Bug Type: Enhancement Format: JSON labels Aug 21, 2022

latk added this to the Upcoming release milestone Aug 21, 2022

update & enable JSON references for "shadow" test

b32de44

latk mentioned this pull request Sep 4, 2022

JSON backwards compatibility #668

Closed

Spacetown reviewed Sep 5, 2022

View reviewed changes

Spacetown mentioned this pull request Sep 9, 2022

GCOVR_EXCL_START applies to line and branch coverage but not function coverage #669

Closed

Spacetown mentioned this pull request Oct 15, 2022

gcovr 51.: AssertionError: assert self.lineno == other.lineno #586

Closed

Spacetown mentioned this pull request Oct 19, 2022

Branches of unknown origin #677

Closed

Spacetown added 3 commits December 4, 2022 22:42

Add review remarks.

aa4722e

Remove outdated documentation about unsupported branch exclusion.

1dcf374

This was introduced with gcovr#644.

Merge commit '7111438d5cfdced67dd46c99b2dab7caede0e62f' into exclusions

87bd007

latk commented Dec 5, 2022

View reviewed changes

Fix merge and CI errors, remove private data in reference files.

cdd95d4

Spacetown approved these changes Dec 5, 2022

View reviewed changes

latk merged commit ad1657d into gcovr:master Dec 5, 2022

Spacetown mentioned this pull request Feb 28, 2023

Filter Option Exclude (-e or --exclude) Does Not Apply To JSON Report? #739

Closed

D-Walther pushed a commit to D-Walther/gcovr that referenced this pull request Jun 16, 2025

Merge pull request gcovr#663 from latk/exclusions

1c1aaba

Fixes + improvements for exclusion processing

Fixes + improvements for exlusion processing #663

Fixes + improvements for exlusion processing #663

Uh oh!

Conversation

latk commented Aug 21, 2022

Uh oh!

codecov bot commented Aug 21, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Spacetown commented Aug 21, 2022

Uh oh!

Spacetown left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Spacetown commented Sep 9, 2022

Uh oh!

latk commented Dec 4, 2022

Uh oh!

latk left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

latk Dec 5, 2022

Choose a reason for hiding this comment

Uh oh!

Spacetown Dec 5, 2022

Choose a reason for hiding this comment

Uh oh!

Spacetown Dec 5, 2022

Choose a reason for hiding this comment

Uh oh!

Spacetown commented Dec 5, 2022

Uh oh!

Spacetown left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

codecov bot commented Aug 21, 2022 •

edited

Loading