|
| 1 | +# Coverage |
| 2 | +utPLSQL comes with build-in coverage reporting engine. The code coverage reporting is based off DBMS_PROFILER package. Code coverage is gathered for the following source types: |
| 3 | +* package bodies |
| 4 | +* type bodies |
| 5 | +* triggers |
| 6 | +* stored procedures |
| 7 | +* stored functions |
| 8 | + |
| 9 | +Note: |
| 10 | + |
| 11 | +> The package specifications and type specifications are explicitly excluded from code coverage analysis.This limitation is introduced to avoid false-negatives. Most of the package specifications don't contain executable code. The only exception is initialization of global constants and variables in package specification.Since, most of package specifications are not executable at all, there is no information available on the number of lines covered and those would eb reported as 0% covered, which is not desired. |
| 12 | +
|
| 13 | +To obtain information about code coverage of your Unit Tests, all you need to do is run your unit tests with one of build-in code coverage reporters. |
| 14 | +Following code coverage reporters are supplied with utPLSQL: |
| 15 | +* `ut_coverage_html_reporter` - generates a HTML coverage report providing summary and detailed information on code coverage. The html reporter is based on open-source [simplecov-html](https://github.com/colszowka/simplecov-html) reporter for Ruby. It includes source code of the code that was covered (if possible) |
| 16 | +* `ut_coveralls_reporter` - generates a JSON coverage report providing detailed information on code coverage with line numbers. This coverage report is designed to be consumed by cloud services like [coveralls](https://coveralls.io) |
| 17 | +* `ut_coverage_sonar_reporter` - generates a JSON coverage report providing detailed information on code coverage with line numbers. This coverage report is designed to be consumed by local services like [sonarqube](https://about.sonarqube.com/) |
| 18 | + |
| 19 | +##Security model |
| 20 | +Code coverage is using DBMS_PROFILER to gather information about execution of code under test and therefore follows the [DBMS_PROFILER's Security Model](https://docs.oracle.com/database/121/ARPLS/d_profil.htm#ARPLS67465) |
| 21 | +In order to be able to gather coverage information, user executing unit tests needs to be either: |
| 22 | +* Owner of the code that is tested |
| 23 | +* Have the following privileges to be able to gather coverage on code owned by other users: |
| 24 | + * `create any procedure` system privilege |
| 25 | + * `execute` privilege on the code that is tested (not only the unit tests) or `execute any procedure` system privilege |
| 26 | + |
| 27 | +If you have `execute` privilege on the code that are tested, but do not have `create any procedure` system privilege, the code that is tested will be reported as not covered (coverage = 0%). |
| 28 | +If you have `execute` privilege only on the unit tests, but do not have `execute` privilege on the code that is tested, the code will not be reported by coverage - as if it did not exist in the database. |
| 29 | +If the code that is testes is complied as NATIVE, the code coverage will not be reported as well. |
| 30 | + |
| 31 | +##Running unite tests with coverage |
| 32 | +Using code coverage functionality is as easy as using any other [reporter](reporters.md) for utPLSQL project. All you need to do is run your tests from your preferred SQL tool and save the outcomes of reporter to a file. |
| 33 | +All you need to do, is pass the constructor of the reporter to your `ut.run` |
| 34 | + |
| 35 | +Example: |
| 36 | +```sql |
| 37 | +begin |
| 38 | + ut.run(ut_coverage_html_reporter()); |
| 39 | +end; |
| 40 | +/ |
| 41 | +``` |
| 42 | +Executes all unit tests in current schema, gather information about code coverage and output the html text into DBMS_OUTPUT. |
| 43 | +The `ut_coverage_html_reporter` will produce a interactive HTML report. You may see a sample of code coverage for utPLSQL project [here](https://jgebal.github.io/utPLSQL-coverage-html/) |
| 44 | + |
| 45 | +The report provides a summary information with list of source code that was expected to be covered. |
| 46 | + |
| 47 | + |
| 48 | + |
| 49 | +The report allow to navigate to every source and inspect line by line coverage. |
| 50 | + |
| 51 | + |
| 52 | + |
| 53 | + |
| 54 | +##Coverage reporting options |
| 55 | +By default the database schema/schemes containing the tests that were executed during the run, are fully reported by coverage reporter. |
| 56 | +All valid unit tests are excluded from the report regardless if they were invoked or not. This way the coverage report is not affected by presence of tests and contains only the tested code. |
| 57 | + |
| 58 | +The default behavior of coverage reporters can be altered, depending on your needs. |
| 59 | + |
| 60 | +###Including/excluding objects in coverage reports |
| 61 | +The most basic options are the include/exclude objects lists. |
| 62 | +You may specify both include and exclude objects lists to specify which objects are to be included in the report and which are to be excluded. |
| 63 | +Both of those options are meant to be used to narrow down the scope of unit test runs, that is broad by default. |
| 64 | + |
| 65 | +Example: |
| 66 | +```sql |
| 67 | +exec ut.run('ut3_user.test_award_bonus', ut_coverage_html_reporter(a_include_object_list=>ut_varchar2_list('ut3_user.award_bonus'))); |
| 68 | +``` |
| 69 | +Executes test `test_award_bonus` and gather coverage only on object `ut3_user.award_bonus` |
| 70 | + |
| 71 | +Alternatively you could run: |
| 72 | +```sql |
| 73 | +exec ut.run('ut3_user.test_award_bonus', ut_coverage_html_reporter(a_exclude_object_list=>ut_varchar2_list('ut3_user.betwnstr'))); |
| 74 | +``` |
| 75 | +Executes test `test_award_bonus` and gather on all objects in schema `ut3_user` except valid unit test objects and object `betwnstr` that schema. |
| 76 | + |
| 77 | +You can also combine the parameters and both will be applied. |
| 78 | + |
| 79 | +###Defining different schema name(s) |
| 80 | +In some architectures, you might end up in a situation, where your unit tests exist in a different schema than the tested code. |
| 81 | +This is not the default or recommended approach but is supporter by utPLSQL. |
| 82 | +In such scenarios, you would probably have a separate database schema to hold unit tests and a separate schema/schemes to hold the tested code. |
| 83 | +Since by default, coverage reporting is done on the schema/schemes that the invoked tests are on, the code will not be included in coverage report as it is in a different schema than the invoked tests. |
| 84 | + |
| 85 | +In this situation you need to provide list of schema names that the tested code is in. This option overrides the default schema names for coverage. |
| 86 | + |
| 87 | +Example: |
| 88 | +```sql |
| 89 | +exec ut.run('ut3_user.test_award_bonus', ut_coverage_html_reporter(a_schema_names=>ut_varchar2_list('usr'))); |
| 90 | +``` |
| 91 | +Executes test `test_award_bonus` in schema `ut3_user` and gather coverage for that execution on all non `unit-test` objects from schema `usr`. |
| 92 | + |
| 93 | +You can combine schema names with include/exclude parameters and all will be applied. |
| 94 | +The `a_schema_names` parameter takes precedence however, so if include list contains objects from other schemes, that will not be considered. |
| 95 | + |
| 96 | +Example: |
| 97 | +```sql |
| 98 | +begin |
| 99 | + ut.run( |
| 100 | + 'ut3_user.test_award_bonus', |
| 101 | + ut_coverage_html_reporter( |
| 102 | + a_schema_names => ut_varchar2_list('usr'), |
| 103 | + a_include_object_list => ut_varchar2_list('usr.award_bonus'), |
| 104 | + a_exclude_object_list => ut_varchar2_list('usr.betwnstr') |
| 105 | + ) |
| 106 | + ); |
| 107 | +end; |
| 108 | +``` |
| 109 | +Executes test `test_award_bonus` in schema `ut3_user` and gather coverage for that execution on `award_bonus` object from schema `usr`. The exclude list is of no relevance as it is not overlapping with include list. |
| 110 | + |
| 111 | +###Working with projects and project files |
| 112 | +Both `sonar` and `coveralls` are utilities that are more project-oriented than database-centric. They report statistics and coverage for project files in version control system. |
| 113 | +Nowadays, most of database projects are moving away from database-centric approach towards project/product-centric approach. |
| 114 | +Coverage reporting of utPLSQL allows you to perform code coverage analysis for your project files. |
| 115 | +This feature is supported by all build-in coverage reporting formats. |
| 116 | + |
| 117 | +When using this invocation syntax, coverage is only reported for the provided files, so using project files as input for coverage is also a way of limiting the scope of coverage analysis. |
| 118 | +This syntax also allows usage of `a_include_object_list` and `a_exclude_object_list` as optional parameters to filter the scope of analysis. |
| 119 | + |
| 120 | + |
| 121 | +**Reporting using externally provided file mapping** |
| 122 | +One of ways to perform coverage reporting on your project files is to provide to the coverage reporter a list of file path/names along with mapping to corresponding object name and object type. |
| 123 | + |
| 124 | +Example: |
| 125 | +```sql |
| 126 | +begin |
| 127 | + ut.run( |
| 128 | + 'usr', |
| 129 | + ut_coverage_html_reporter( |
| 130 | + a_file_mappings=>ut_coverage_file_mappings( |
| 131 | + ut_coverage_file_mapping( |
| 132 | + file_name => 'sources/hr/award_bonus.prc', |
| 133 | + object_owner => 'usr', |
| 134 | + object_name => 'award_bonus', |
| 135 | + object_type => 'procedure' |
| 136 | + ), |
| 137 | + ut_coverage_file_mapping( |
| 138 | + file_name => 'sources/hr/betwnstr.fnc', |
| 139 | + object_owner => 'usr', |
| 140 | + object_name => 'betwnstr', |
| 141 | + object_type => 'function' |
| 142 | + ) |
| 143 | + ) |
| 144 | + ) |
| 145 | + ); |
| 146 | +end; |
| 147 | +``` |
| 148 | + |
| 149 | +Executes all tests in schema `usr` and reports coverage for that execution on procedure `award_bonus` and function `betwnstr`. The coverage report is mapped-back to file-system object names with paths. |
| 150 | + |
| 151 | +**Reporting using regex file mapping rule** |
| 152 | +If file names and paths in your project follow a well established naming conventions, |
| 153 | +then you can use the predefined rule for mapping file names to object names or you can define your own rule and pass it to the coverage reporter at runtime. |
| 154 | + |
| 155 | +Example of running with predefined regex mapping rule. |
| 156 | +```sql |
| 157 | +begin |
| 158 | + ut.run( |
| 159 | + 'usr', |
| 160 | + ut_coverage_html_reporter( |
| 161 | + a_file_paths => ut_varchar2_list('sources/hr/award_bonus.prc','sources/hr/betwnstr.fnc') |
| 162 | + ) |
| 163 | + ); |
| 164 | +end; |
| 165 | +``` |
| 166 | + |
| 167 | +The predefined rule is based on the following default values for parameters: |
| 168 | +* `a_regex_pattern => '.*(\\|\/)((\w+)\.)?(\w+)\.(\w{3})'` |
| 169 | +* `a_object_owner_subexpression => 3` |
| 170 | +* `a_object_name_subexpression => 4` |
| 171 | +* `a_object_type_subexpression => 5` |
| 172 | +* `a_file_to_object_type_mapping` - defined in table below |
| 173 | + |
| 174 | +The predefined file extension to object type mappings |
| 175 | + |
| 176 | +| file extension | object type | |
| 177 | +| -------------- | ----------- | |
| 178 | +| tpb | type body | |
| 179 | +| pkb | package body | |
| 180 | +| bdy | package body | |
| 181 | +| trg | trigger | |
| 182 | +| fnc | function | |
| 183 | +| prc | procedure | |
| 184 | + |
| 185 | +Since package specification and type specifications are not considered by coverage, the file extensions for those objects are not included in the mapping. |
| 186 | + |
| 187 | +Examples of filename paths that will be mapped correctly using predefined rules. |
| 188 | +* `[...]directory[/subdirectory[/...]]/object_name.(tpb|pkb|trg|fnc|prc)` |
| 189 | +* `[...]directory[/subdirectory[/...]]/schema_name.object_name.(tpb|pkb|trg|fnc|prc)` |
| 190 | +* `[...]directory[\subdirectory[\...]]\object_name.(tpb|pkb|trg|fnc|prc)` |
| 191 | +* `[...]directory[\subdirectory[\...]]\schema_name.object_name.(tpb|pkb|trg|fnc|prc)` |
| 192 | + |
| 193 | +If file names in your project structure are not prefixed with schema name (like above), the coverage report will look for objects to match the file names in the `current schema` of the connection that was used to execute tests with coverage. |
| 194 | +If for whatever reasons you use a user and current schema that is different then schem that holds your project code, you should use `a_schema_name` parameter to inform coverage reporter about database schema to be used for object lookup. |
| 195 | + |
| 196 | +Example: |
| 197 | +```sql |
| 198 | +begin |
| 199 | + ut.run( |
| 200 | + 'usr', |
| 201 | + ut_coverage_html_reporter( |
| 202 | + a_schema_name => 'hr', |
| 203 | + a_file_paths => ut_varchar2_list('sources/hr/award_bonus.prc','sources/hr/betwnstr.fnc') |
| 204 | + ) |
| 205 | + ); |
| 206 | +end; |
| 207 | +``` |
| 208 | + |
| 209 | +If your project structure is different, you may define your own mapping rule using regex. |
| 210 | + |
| 211 | +Example: |
| 212 | +```sql |
| 213 | +begin |
| 214 | + ut.run( |
| 215 | + 'usr', |
| 216 | + ut_coverage_html_reporter( |
| 217 | + a_file_paths => ut_varchar2_list('sources/hr/procedures/award_bonus.sql','sources/hr/functions/betwnstr.sql'), |
| 218 | + a_regex_pattern => '.*(\\|\/)(\w+)\.(\w+)\.(\w{3})', |
| 219 | + a_object_owner_subexpression => 2, |
| 220 | + a_object_type_subexpression => 3, |
| 221 | + a_object_name_subexpression => 4, |
| 222 | + a_file_to_object_type_mapping => ut_key_value_pairs( |
| 223 | + ut_key_value_pair('functions', 'function'), |
| 224 | + ut_key_value_pair('procedures', 'procedure') |
| 225 | + ) |
| 226 | + ); |
| 227 | +end; |
| 228 | +``` |
| 229 | + |
0 commit comments