Annotations are used to configure tests and suites in a declarative way similar to modern OOP languages. This way, test configuration is stored along with the test logic inside the test package.

No additional configuration files or tables are needed for test cases. The annotation names are based on popular testing frameworks such as JUnit.

The framework runner searches for all the suitable annotated packages, automatically configures suites, forms the suite hierarchy, executes it and reports results in specified formats.

There can be many annotations for a procedure.

Valid procedure annotations example:

package test_package is

--%suite


The following are best practices we at utPLSQL have learned about PL/SQL and Unit Testing.


utPLSQL comes with a built-in coverage reporting engine. The code coverage reporting uses package DBMS_PROFILER (and DBMS_PLSQL_CODE_COVERAGE on Oracle database version 12.2 and above) provided with Oracle database.

Code coverage is gathered for the following source types:

* package bodies

* type bodies

* triggers

To obtain information about code coverage for unit tests, run utPLSQL with one of built-in code coverage reporters.

The following code coverage reporters are supplied with utPLSQL:

* `ut_coverage_html_reporter` - generates a HTML coverage report providing summary and detailed information on code coverage. The HTML reporter is based on the open-source [simplecov-html](https://github.com/colszowka/simplecov-html) reporter for Ruby. It includes source code of the code that was covered (if the code is accessible for test user)

* `ut_coveralls_reporter` - generates a [Coveralls compatible JSON](https://coveralls.zendesk.com/hc/en-us/articles/201774865-API-Introduction) coverage report providing detailed information on code coverage with line numbers. This coverage report is designed to be consumed by cloud services like [Coveralls](https://coveralls.io)

* `ut_coverage_sonar_reporter` - generates a [Sonar Compatible XML](https://docs.sonarqube.org/latest/analysis/generic-test/) coverage report providing detailed information on code coverage with line numbers. This coverage report is designed to be consumed by services like [SonarQube](https://www.sonarqube.org/) and [SonarCloud](https://about.sonarcloud.io/)

utPLSQL code coverage uses DBMS_PROFILER to gather information about the execution of code under test and therefore follows the [DBMS_PROFILER's Security Model](https://docs.oracle.com/database/121/ARPLS/d_profil.htm#ARPLS67465).

In order to be able to gather coverage information, the user executing unit tests needs to be either:

* The owner of the code that is being tested

* Have the following privileges to be able to gather coverage on code owned by other users:

* `create any procedure` system privilege

**Important notes**

The order of priority is for evaluation of include/exclude filter parameters is as follows.

- if `a_source_file_mappings` is defined then all include/exclude parameters are ignored (see section below for usage of `a_source_file_mappings` parameter )

- else if `a_include_schema_expr` or `a_include_object_expr` parameter is specified then parameters `a_coverage_schemes` and `a_include_objects` are ignored

- else if `a_include_objects` is specified then the coverage is gathered only on specified database objects.

To be able to effectively use reporters dedicated for those tools, utPLSQL provides functionality for mapping database object names to project files.

There are a few significant differences when running coverage on project files compared to running coverage on schema(s).

- Coverage is only reported on objects that were successfully mapped to project files.

- Project files (database objects) that were not executed at all are not reported as fully uncovered. It is up to the consumer (Sonar/Coveralls) to determine if project file should be considered as 0% coverage or just ignored.

```

By default, utPLSQL will convert file paths into database objects using the following regular expression `/(((\w|[$#])+)\.)?((\w|[$#])+)\.(\w{3})$`

- object owner (if it is present) is identified by the expression in the second set of brackets

- object name is identified by the expression in the fourth set of brackets

- object type is identified by the expression in the sixth set of brackets


The utPLSQL is responsible for handling exceptions wherever they occur in the test run. utPLSQL is trapping most of the exceptions so that the test execution is not affected by individual tests or test packages throwing an exception.

The framework provides a full stacktrace for every exception that was thrown. The stacktrace is clean and does not include any utPLSQL library calls in it.

To achieve rerunability, the package state invalidation exceptions (ORA-04068, ORA-04061) are not handled and test execution will be interrupted if such exceptions are encountered. This is because of how Oracle behaves on those exceptions.

Test execution can fail for different reasons. The failures on different exceptions are handled as follows:

* A test package without body - each `--%test` is reported as failed with exception, nothing is executed

* A test package with _invalid body_ - each `--%test` is reported as failed with exception, nothing is executed

* A test package with _invalid spec_ - package is not considered a valid unit test package and is excluded from execution. When trying to run a test package with invalid spec explicitly, exception is raised. Only valid specifications are parsed for annotations


Validation of the code under test (the tested logic of procedure/function etc.) is performed by comparing the actual data against the expected data.

utPLSQL uses expectations and matchers to perform the check on the data.

**Note:**

There are two ways to use expectations:

- by invoking utPLSQL framework to execute suite(s) of utPLSQL tests

- without invoking the utPLSQL framework - running expectations standalone

When expectations are ran as a part of a test suite, the framework tracks:

- status of each expectation

- outcomes (messages) produced by each expectation

**Note:**

utPLSQL provides the following matchers to perform checks on the expected and actual values.

- `be_between( a_upper_bound {data-type}, a_lower_bound {data-type} )`

Since NULL is neither *true* nor *false*, both expectations will report failure.

The matrix below illustrates the data types supported by different matchers.

| **be_within().of_()** | | | | X | X | X | X | X | | | | | | | |

| **be_within_pct().of_()** | | | | | X | | | | | | | | | | |

Testing is not limited to checking for happy-path scenarios. When writing tests, you often want to validate that in specific scenarios, an exception is thrown.

For more details see documentation of the [`--%throws` annotation.](annotations.md#throws-annotation)

You can choose different matchers to validate that your PL/SQL code is working as expected.

Below is an example of building a simple function with TDD.

We have a requirement to build a function that will return a substring of a string that is passed to the function.

- start_position

- end_position

We will start from the bare minimum and move step by step, executing tests every time we make minimal progress.

This way, we assure we don't jump ahead too much and produce code that is untested or untestable.

create or replace package test_betwnstr as

create or replace package test_betwnstr as

Well our test is failing as the package specification requires a body.

create or replace package body test_betwnstr as

Our test is failing as the test suite package body is invalid.

Looks like we need to define the function we want to test.

create or replace function betwnstr( a_string varchar2, a_start_pos integer, a_end_pos integer ) return varchar2

So now we see that our test works but the function does not return the expected results.

Let us fix this and continue from here.

The function returned a string one character short, so we need to add 1 to the substr parameter.

So our test is now passing, great!

Once our tests are passing, we can safely refactor (restructure) the code as we have a safety harness

in place to ensure that after the restructuring and cleanup of the code, everything is still working.

One thing worth mentioning is that refactoring of tests is as important as refactoring of code. Maintainability of both is equally important.

It seems like our work is done. We have a function that returns a substring from start position to end position.

As we move through the process of adding tests, it's very important to think about edge cases.

We should define expected behavior for each of these edge cases.

Once defined we can start implementing tests for those behaviors and adjust the tested function to meet the requirements specified in the tests.

A new requirement was added:

Start position zero - should be treated as start position one

Looks like our function does not work as expected for zero start position.

Let's fix our function so that the new requirement is met

Great! We have made some visible progress.

When all tests are passing we can proceed with a safe cleanup of our code.

You may continue on with the remaining edge cases from here.