Thanks to visit codestin.com
Credit goes to github.com

Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Coverage documentation improvements.
Fixed datatype for ut_coverage_options.tps attributes
Formatting.
  • Loading branch information
jgebal committed Mar 1, 2022
commit bd05e4cb19ea7a77ad1bc629124e58a714c13226
122 changes: 78 additions & 44 deletions docs/userguide/coverage.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
![version](https://img.shields.io/badge/version-v3.1.13.3908--develop-blue.svg)

# Coverage
utPLSQL comes with a built-in coverage reporting engine. The code coverage reporting uses DBMS_PROFILER package provided with Oracle database.
utPLSQL comes with a built-in coverage reporting engine. The code coverage reporting uses package DBMS_PROFILER (and DBMS_PLSQL_CODE_COVERAGE on Oracle database version 12.2 and above) provided with Oracle database.
Code coverage is gathered for the following source types:
* package bodies
* type bodies
Expand All @@ -15,7 +15,7 @@ Code coverage is gathered for the following source types:

To obtain information about code coverage for unit tests, run utPLSQL with one of built-in code coverage reporters.
The following code coverage reporters are supplied with utPLSQL:
* `ut_coverage_html_reporter` - generates a HTML coverage report providing summary and detailed information on code coverage. The HTML reporter is based on the open-source [simplecov-html](https://github.com/colszowka/simplecov-html) reporter for Ruby. It includes source code of the code that was covered (if possible)
* `ut_coverage_html_reporter` - generates a HTML coverage report providing summary and detailed information on code coverage. The HTML reporter is based on the open-source [simplecov-html](https://github.com/colszowka/simplecov-html) reporter for Ruby. It includes source code of the code that was covered (if the code is accessible for test user)
* `ut_coveralls_reporter` - generates a [Coveralls compatible JSON](https://coveralls.zendesk.com/hc/en-us/articles/201774865-API-Introduction) coverage report providing detailed information on code coverage with line numbers. This coverage report is designed to be consumed by cloud services like [Coveralls](https://coveralls.io)
* `ut_coverage_sonar_reporter` - generates a [Sonar Compatible XML](https://docs.sonarqube.org/latest/analysis/generic-test/) coverage report providing detailed information on code coverage with line numbers. This coverage report is designed to be consumed by services like [SonarQube](https://www.sonarqube.org/) and [SonarCloud](https://about.sonarcloud.io/)
* `ut_coverage_cobertura_reporter` - generates a basic Cobertura coverage (http://cobertura.sourceforge.net/xml/coverage-04.dtd) report providing detailed information on code coverage with line numbers. This coverage report is designed to be consumed by services like TFS and Jenkins. Check this link for an example of XML generated by Java: https://raw.githubusercontent.com/jenkinsci/cobertura-plugin/master/src/test/resources/hudson/plugins/cobertura/coverage-with-data.xml
Expand All @@ -30,7 +30,7 @@ In order to be able to gather coverage information, the user executing unit test

If you have `execute` privilege on the code that is being tested, but do not have `create any procedure` system privilege, then the code that is being tested will be reported as not covered (coverage = 0%).
If you have `execute` privilege only on the unit tests, but do not have `execute` privilege on the code that is being tested, the code will not be reported by coverage - as if it did not exist in the database.
If the code that is being tested is complied as NATIVE, the code coverage will not be reported as well.
If the code that is being tested is compiled as NATIVE, the code coverage will not be reported as well.

## Running unit tests with coverage
Using the code coverage functionality is as easy as using any other [reporter](reporters.md) for the utPLSQL test-run. You just run your tests from your preferred SQL tool and save the reporter results to a file.
Expand Down Expand Up @@ -65,6 +65,7 @@ Due to security model of `dbms_plsql_code_coverage` package, utPLSQL grants acce
The access and synonyms will be public when using the headless installation. This approach avoids complexity of forcing every user of utPLSQL framework to create tables on their own.

Sample output:

![Package Coverage Summary](../images/extended_coverage_html_summary.png)

![Line Coverage Details](../images/extended_coverage_html_line.png)
Expand Down Expand Up @@ -128,42 +129,80 @@ exec ut.run('unit_test_schema', ut_coverage_html_reporter(), a_coverage_schemes

#### Filtering objects in coverage reports

There are six options that can be used to narrow down the scope of coverage report:
- `a_include_objects` - list of `[object_owner.].object_name` to be included in the coverage report.
- `a_exclude_objects` - list of `[object_owner.].object_name` to be excluded from the coverage report
- `a_include_schema_expr` - string of regex expression of schemas to be included in the coverage report. It will override `a_include_objects` for example :
```sql
a_include_schema_expr => '^ut3_develop',
a_include_objects => ut3_develop.ut_varchar2_list( 'ut3_tester_helper.regex_dummy_cov' )
```
Will result in showing coverage for all schemas that match regular expression `^ut3_develop` and ignore objectes defined by parameter `a_include_objects`

- `a_include_object_expr` - string of regex expression of objects ( no schema names included ) to be included in the coverage report. It will override `a_include_objects` for example:
```sql
a_include_object_expr => 'regex123',
a_include_objects => ut3_develop.ut_varchar2_list( 'ut3_tester_helper.regex_dummy_cov' )
```
Will result in showing coverage for all objects that name match regular expression `regex123` and ignore objectes defined by parameter `a_include_objects`

- `a_exclude_schema_expr` - string of regex expression of schemas to be excluded from the coverage report. It will override `a_exclude_objects` for example:
```sql
a_exclude_schema_expr => '^ut3_tester',
a_exclude_objects => ut3_develop.ut_varchar2_list( 'ut3_develop.regex_dummy_cov' )
```
Will result in showing coverage for all objects that schema is not matching regular expression `^ut3_tester` and ignore exclusion defined by parameter `a_exclude_objects`

- `a_exclude_object_expr` - string of regex expression of objects to be excluded from the coverage report. It will override `a_exclude_objects` for example
```sql
a_exclude_object_expr => 'regex123',
a_exclude_objects => ut3_develop.ut_varchar2_list( 'ut3_develop.regex_dummy_cov' )
```
Will result in showing coverage for all objects that name is not matching regular expression `regex123` and ignore exclusion defined by parameter `a_exclude_objects`

There are multiple parameters that can be used to define the scope of coverage report:
- `a_source_file_mappings ( ut_file_mappings )` - map of filenames to database objects. It is used for file-based coverage - see below.
- `a_include_schema_expr ( varchar(4000) )` - string of regex expression of schemas to be included in the coverage report.
- `a_include_object_expr ( varchar(4000) )` - string of regex expression of objects ( without schema name ) to be included in the coverage report.
- `a_exclude_schema_expr ( varchar(4000) )` - string of regex expression of schemas to be excluded from the coverage report.
- `a_exclude_object_expr ( varchar(4000) )` - string of regex expression of objects to be excluded from the coverage report.
- `a_coverage_schemes ( ut_varchar2_list )` - List of database schema names to gather coverage on.
- `a_include_objects ( ut_varchar2_list )` - list of `[object_owner.]object_name` to be included in the coverage report.
- `a_exclude_objects ( ut_varchar2_list )` - list of `[object_owner.]object_name` to be excluded from the coverage report.

You may specify both _include_ and _exclude_ options to gain more control over what needs to be included / excluded from the coverage report.

The object owner is optional in the object list.
If you do not provide an object owner, the _include_/_exclude_ list will be considered for every schema used for coverage gathering (as described above).
**Important notes**
The order of priority is for evaluation of include/exclude filter parameters is as follows.
- if `a_source_file_mappings` is defined then all include/exclude parameters are ignored (see section below for usage of `a_source_file_mappings` parameter )
- else if `a_include_schema_expr` or `a_include_object_expr` parameter is specified then parameters `a_coverage_schemes` and `a_include_objects` are ignored
- else if `a_include_objects` is specified then the coverage is gathered only on specified database objects.
- if `a_coverage_schemes` is specified then those schemas are used for objects in `a_include_objects` without schema name
- if `a_coverage_schemes` is not specified then schema from paths (`a_paths`) parameter are used for objects in `a_include_objects` without schema name
- else if, only the `a_coverage_schemes` is specified then the coverage is gathered only on specified database schemas
- else if no coverage specific parameters are provided coverage is gathered on all schemas specified in paths passed to run procedure
- else if no paths were specified, the coverage is gathered on current schema of the session running the tests


Example: Limiting coverage by schema regex.
```sql
begin
ut.run(ut_varchar2_list('user_1','user_2'), ut_coverage_html_reporter(),
a_include_schema_expr => '^ut3_develop'
);
end;
```
Will result in showing coverage for all schemas that match regular expression `^ut3_develop`

```sql
begin
ut.run(ut_varchar2_list('user_1','user_2'), ut_coverage_html_reporter(),
a_include_schema_expr => '^ut3_develop', a_include_objects => ut_varchar2_list( 'ut3_tester_helper.regex_dummy_cov' )
);
end;
```
Will result in showing coverage for all schemas that match regular expression `^ut3_develop`.
Will ignore the `a_include_objects` parameter

```sql
begin
ut.run(ut_varchar2_list('user_1','user_2'), ut_coverage_html_reporter(),
a_include_object_expr => 'regex123', a_include_objects => ut_varchar2_list( 'ut3_tester_helper.regex_dummy_cov' )
);
end;
```
Will result in showing coverage for all objects that name match regular expression `regex123`.
Will ignore the `a_include_objects` parameter.

```sql
begin
ut.run(ut_varchar2_list('user_1','user_2'), ut_coverage_html_reporter(),
a_exclude_schema_expr => '^ut3_tester', a_exclude_objects => ut_varchar2_list( 'regex_dummy_cov' )
);
end;
```
Will result in showing coverage for objects in all schema except schemas that are matching regular expression `^ut3_tester`
Will ignore exclusion defined by parameter `a_exclude_objects`

```sql
begin
ut.run(ut_varchar2_list('user_1','user_2'), ut_coverage_html_reporter(),
a_exclude_object_expr => 'regex123', a_exclude_objects => ut_varchar2_list( 'regex_dummy_cov' )
);
end;
```
Will result in showing coverage for all objects that name is not matching regular expression `regex123`.
Will ignore exclusion defined by parameter `a_exclude_objects`.


Example: Limiting coverage by object name, for tested code located in the same schema as the unit tests.
```sql
Expand Down Expand Up @@ -214,7 +253,8 @@ Executes test `test_award_bonus` in schema `unit_test_schema`. Coverage will be

**Note**
> When running coverage on schema objects, all source code of package bodies, functions, procedures, type bodies and triggers that were not executed will be reported as having 0% code coverage and all source code lines will show as uncovered.
> This is different than the behavior when gathering coverage on project files.
> This is different from the behavior when gathering coverage on project files.


### Project based Coverage

Expand Down Expand Up @@ -687,7 +727,7 @@ begin
end;
```

## Reporting coverage outside of utPLSQL
## Reporting coverage outside utPLSQL and in parallel sessions

utPSLQL allows fo standalone reporting code coverage across multiple database sessions. This functionality enables coverage reporting for external testing tools.

Expand Down Expand Up @@ -811,9 +851,3 @@ If the identifier is not unique, previous runs of coverage that used the same `c
For details on the meaning of `schema_names`, `exclude_objects`, `include_objects`, `file_mappings` see sections above.
Note that data-types of include/exclude/schema lists are different when calling `ut.run` vs. calling `get_report/get_report_cursor`.

The order of priority is for evaluation of include/exclude is as follows.
- if `file_mappings` is specified then all include/exclude parameters are ignored
- else if `..._expr` is specified then include_objects/exclude_objects parameters are ignored
- else if `schema_names` is specified then the coverage is gathered on all object of specified schemas
- else coverage is gathered on all schemas specified in paths passed to run procedure
- if no paths were specified, the coverage is gathered on current schema of the session running the tests
21 changes: 9 additions & 12 deletions source/core/coverage/ut_coverage.pkb
Original file line number Diff line number Diff line change
Expand Up @@ -180,21 +180,18 @@ create or replace package body ut_coverage is
ut_event_manager.trigger_event(ut_event_manager.gc_debug, ut_key_anyvalues().put('l_sql',l_sql) );

if a_coverage_options.file_mappings is not empty then
open l_cursor for l_sql using a_coverage_options.file_mappings,a_coverage_options.exclude_schema_expr,
a_coverage_options.exclude_object_expr,l_excluded_objects,
l_skip_objects;
open l_cursor for l_sql using a_coverage_options.file_mappings, a_coverage_options.exclude_schema_expr,
a_coverage_options.exclude_object_expr, l_excluded_objects, l_skip_objects;
elsif coalesce(a_coverage_options.include_schema_expr,a_coverage_options.include_object_expr) is not null then
open l_cursor for l_sql using a_coverage_options.include_schema_expr,a_coverage_options.include_object_expr,
a_coverage_options.exclude_schema_expr,a_coverage_options.exclude_object_expr,
l_excluded_objects,l_skip_objects;
open l_cursor for l_sql using a_coverage_options.include_schema_expr, a_coverage_options.include_object_expr,
a_coverage_options.exclude_schema_expr, a_coverage_options.exclude_object_expr,
l_excluded_objects, l_skip_objects;
elsif a_coverage_options.include_objects is not empty then
open l_cursor for l_sql using a_coverage_options.include_objects,a_coverage_options.exclude_schema_expr,
a_coverage_options.exclude_object_expr,l_excluded_objects,
l_skip_objects;
open l_cursor for l_sql using a_coverage_options.include_objects, a_coverage_options.exclude_schema_expr,
a_coverage_options.exclude_object_expr, l_excluded_objects, l_skip_objects;
else
open l_cursor for l_sql using a_coverage_options.schema_names,a_coverage_options.exclude_schema_expr,
a_coverage_options.exclude_object_expr,l_excluded_objects,
l_skip_objects;
open l_cursor for l_sql using a_coverage_options.schema_names, a_coverage_options.exclude_schema_expr,
a_coverage_options.exclude_object_expr, l_excluded_objects, l_skip_objects;
end if;
return l_cursor;
end;
Expand Down
8 changes: 4 additions & 4 deletions source/core/types/ut_coverage_options.tps
Original file line number Diff line number Diff line change
Expand Up @@ -21,10 +21,10 @@ create or replace type ut_coverage_options force as object (
exclude_objects ut_object_names,
include_objects ut_object_names,
file_mappings ut_file_mappings,
include_schema_expr varchar2(32767),
include_object_expr varchar2(32767),
exclude_schema_expr varchar2(32767),
exclude_object_expr varchar2(32767),
include_schema_expr varchar2(4000),
include_object_expr varchar2(4000),
exclude_schema_expr varchar2(4000),
exclude_object_expr varchar2(4000),
constructor function ut_coverage_options(
self in out nocopy ut_coverage_options,
coverage_run_id raw,
Expand Down