Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Commit 2135bd6

Browse files
jgebaljgebal
committed
Updated documentation for --%throws annotation.
Updated documentation to assure that annotations are mentioned as `--%annotation` rather than `%annotation` Fixed bug, where `--%throws(-00001)` would not catch exception `ORA-00001` due to leading zero. Added full stack trace, when a different than expected exception was thrown. Added ut_integer_list_type. Cleanup of `ut_executable_test` Reworked tests for `--%throws` to check for the test report text content.
1 parent d02c406 commit 2135bd6

17 files changed

Lines changed: 307 additions & 234 deletions

‎docs/userguide/annotations.md‎

Lines changed: 128 additions & 73 deletions
Original file line numberDiff line numberDiff line change
@@ -4,64 +4,101 @@ Annotations are used to configure tests and suites in a declarative way similar
44
No configuration files or tables are needed. The annotation names are based on popular testing frameworks such as jUnit.
55
The framework runner searches for all the suitable annotated packages, automatically configures suites, forms the suite hierarchy, executes it and reports results in specified formats.
66

7-
Annotations are interpreted only in the package specification and are case-insensitive. It is recommended however, to use lower-case annotations as described in this documentation.
7+
Annotations are interpreted only in the package specification and are case-insensitive. We strongly recommend using lower-case annotations as described in this documentation.
88

9-
There are two places where annotations may appear:
9+
There are two locations where annotations can be placed:
10+
- Package level annotations can be placed at the very top of the package specification (`--%suite`, `--%suitepath` etc.)
11+
- Procedure level annotations can be placed right before a procedure (`--%test`, `--%beforeall`, `--%beforeeach` etc.)
12+
13+
If procedure level annotation is not placed right before procedure, it is not considered an annotation for procedure.
14+
15+
Example of invalid procedure level annotations
16+
```sql
17+
create or replace package test_pkg is
18+
19+
--%suite(Name of suite)
20+
21+
--%test
22+
-- this single-line comment makes the TEST annotation no longer associated with the procedure
23+
procedure first_test;
24+
25+
--%test
26+
--procedure some_test; /* This TEST annotation is not associated with any procedure*/
27+
28+
--%test(Name of another test)
29+
procedure another_test;
30+
31+
--%test
32+
/**
33+
* this multi-line comment makes the TEST annotation no longer associated with the procedure
34+
*/
35+
procedure yet_another_test;
36+
end test_pkg;
37+
```
38+
Procedure annotations are defined right before the procedure they reference, no empty lines are allowed, no comment lines can exist between annotation and the procedure.
1039

11-
- at the beginning of the package specification (`%suite`, `%suitepath` etc.)
12-
- right before a procedure (`%test`, `%beforeall`, `%beforeeach` etc.)
1340

1441
Package level annotations need to be separated by at least one empty line from the underlying procedure annotations.
1542

16-
Procedure annotations are defined right before the procedure they reference, no empty lines are allowed.
43+
Example of invalid package level annotation.
44+
```sql
45+
create or replace package test_pkg is
46+
--%suite(Name of suite)
47+
--%test
48+
procedure first_test;
49+
end test_pkg;
50+
```
1751

18-
If a package specification contains the `%suite` annotation, it is treated as a test package and is processed by the framework.
52+
If a package specification contains the `--%suite` annotation, it is treated as a test package and is processed by the framework.
1953

20-
Some annotations accept parameters like `%suite`, `%test` and `%displayname`. The parameters for annotations need to be placed in brackets. Values for parameters should be provided without any quotation marks.
54+
Some annotations accept parameters like `--%suite`, `--%test` and `--%displayname`. The parameters for annotations need to be placed in brackets.
55+
Values for parameters should be provided without any quotation marks.
56+
If the parameters are placed without brackets or with incomplete brackets, they will be ignored.
57+
Example: `--%suite(The name of suite without closing bracket`
2158

2259
# <a name="example"></a>Example of an annotated test package
2360

2461
```sql
2562
create or replace package test_pkg is
2663

27-
-- %suite(Name of suite)
28-
-- %suitepath(all.globaltests)
64+
--%suite(Name of suite)
65+
--%suitepath(all.globaltests)
2966

30-
-- %beforeall
67+
--%beforeall
3168
procedure global_setup;
3269

33-
-- %afterall
70+
--%afterall
3471
procedure global_cleanup;
3572

3673
/* Such comments are allowed */
3774

38-
-- %test
39-
-- %displayname(Name of a test)
40-
-- %throws(-20145,-20146,-20189,-20563)
75+
--%test
76+
--%displayname(Name of a test)
77+
--%throws(-20145,-20146,-20189,-20563)
4178
procedure some_test;
4279

43-
-- %test(Name of another test)
44-
-- %beforetest(setup_another_test)
45-
-- %aftertest(cleanup_another_test)
80+
--%test(Name of another test)
81+
--%beforetest(setup_another_test)
82+
--%aftertest(cleanup_another_test)
4683
procedure another_test;
4784

48-
-- %test
49-
-- %displayname(Name of test)
50-
-- %disabled
85+
--%test
86+
--%displayname(Name of test)
87+
--%disabled
5188
procedure disabled_test;
5289

53-
-- %test(Name of test)
54-
-- %rollback(manual)
90+
--%test(Name of test)
91+
--%rollback(manual)
5592
procedure no_transaction_control_test;
5693

5794
procedure setup_another_test;
5895

5996
procedure cleanup_another_test;
6097

61-
-- %beforeeach
98+
--%beforeeach
6299
procedure test_setup;
63100

64-
-- %aftereach
101+
--%aftereach
65102
procedure test_cleanup;
66103

67104
end test_pkg;
@@ -75,7 +112,7 @@ end test_pkg;
75112
| `%suitepath(<path>)` | Package | Similar to java package. The annotation allows logical grouping of suites into hierarchies. |
76113
| `%displayname(<description>)` | Package/procedure | Human-readable and meaningful description of a suite/test. `%displayname(Name of the suite/test)`. The annotation is provided for flexibility and convenience only. It has exactly the same meaning as `<description>` in `test` and `suite` annotations. If description is provided using both `suite`/`test` and `displayname`, then the one defined as last takes precedence. |
77114
| `%test(<description>)` | Procedure | Denotes that the annotated procedure is a unit test procedure. Optional test description can by provided (see `displayname`). |
78-
| `%throws(<exception_number>)`| Test Procedure | Denotes that the annotated test procedure must throws one of the exception numbers written. If there are no valid number the annotation is ignored. Only applies to test procedures. |
115+
| `%throws(<exception_number>[,<exception_number>[,...]])`| Procedure | Denotes that the annotated procedure must throw one of the exception numbers provided. If no valid numbers were provided as annotation parameters the annotation is ignored. Applicable to test procedures only. |
79116
| `%beforeall` | Procedure | Denotes that the annotated procedure should be executed once before all elements of the suite. |
80117
| `%afterall` | Procedure | Denotes that the annotated procedure should be executed once after all elements of the suite. |
81118
| `%beforeeach` | Procedure | Denotes that the annotated procedure should be executed before each `%test` procedure in the suite. |
@@ -105,17 +142,17 @@ The `%suitepath` annotation is used for such grouping. Even though test packages
105142
```sql
106143
create or replace package test_payment_recognition as
107144

108-
-- %suite(Payment recognition tests)
109-
-- %suitepath(payments)
145+
--%suite(Payment recognition tests)
146+
--%suitepath(payments)
110147

111-
-- %test(Recognize payment by policy number)
148+
--%test(Recognize payment by policy number)
112149
procedure test_recognize_by_num;
113150

114-
-- %test
115-
-- %displayname(Recognize payment by payment purpose)
151+
--%test
152+
--%displayname(Recognize payment by payment purpose)
116153
procedure test_recognize_by_purpose;
117154

118-
-- %test(Recognize payment by customer)
155+
--%test(Recognize payment by customer)
119156
procedure test_recognize_by_customer;
120157

121158
end test_payment_recognition;
@@ -125,14 +162,14 @@ And payments set off test package:
125162
```sql
126163
create or replace package test_payment_set_off as
127164

128-
-- %suite(Payment set off tests)
129-
-- %suitepath(payments)
165+
--%suite(Payment set off tests)
166+
--%suitepath(payments)
130167

131-
-- %test(Set off creation test)
168+
--%test(Set off creation test)
132169
procedure test_create_set_off;
133170

134-
-- %test
135-
-- %displayname(Set off annulation test)
171+
--%test
172+
--%displayname(Set off annulation test)
136173
procedure test_annulate_set_off;
137174

138175
end test_payment_set_off;
@@ -145,12 +182,12 @@ An additional advantage of such grouping is the fact that every element level of
145182
```sql
146183
create or replace package payments as
147184

148-
-- %suite(Payments)
185+
--%suite(Payments)
149186

150-
-- %beforeall
187+
--%beforeall
151188
procedure set_common_payments_data;
152189

153-
-- %afterall
190+
--%afterall
154191
procedure reset_common_paymnets_data;
155192

156193
end payments;
@@ -169,7 +206,7 @@ In general, your unit tests should not use transaction control as long as the co
169206
Keeping the transactions uncommitted allows your changes to be isolated and the execution of tests does not impact others who might be using a shared development database.
170207

171208
If you are in a situation where the code you are testing uses transaction control (common case with ETL code), then your tests probably should not use the default automatic transaction control.
172-
In that case use the annotation `-- %rollback(manual)` on the suite level to disable automatic transaction control for the entire suite.
209+
In that case use the annotation `--%rollback(manual)` on the suite level to disable automatic transaction control for the entire suite.
173210
If you are using nested suites, you need to make sure that the entire suite all the way to the root is using manual transaction control.
174211

175212
It is possible with utPLSQL to change the transaction control on individual suites or tests that are part of complex suite.
@@ -240,70 +277,88 @@ exec ut_runner.purge_cache('HR', 'PACKAGE');
240277

241278
# Throws annotation
242279

243-
utPLSQL uses the `%throws` annotation to allow the user to delimit which exception numbers a test must throws.
244-
245-
If `%throws` is specified for one test and nothing is raise or a different exception than the annotated one is thrown, the test is marked as failed.
280+
The `--%throws` annotation allows you to specify a list of exception numbers that can be expected from a test.
246281

247-
The framework discards the bad arguments, `--%throws(7894562, operaqk, -=1, -20496, pow74d, posdfk3)` it converts that to this `--%throws(-20496)`.
248-
If the arguments is empty `--%throws()`,`--%throws`, or only no valid arguments are provided `--%throws(abe, 723pf)` the annotation is ignored.
282+
If `--%throws(-20001,-20002)` is specified and no exception is raised or the exception raised is not on the list of provided exception numbers, the test is marked as failed.
249283

250-
Example how to use the annotation:
284+
The framework ignores bad arguments. `--%throws(7894562, operaqk, -=1, -20496, pow74d, posdfk3)` will be interpreted as `--%throws(-20496)`.
285+
The annotation is ignored, when no valid arguments are provided `--%throws()`,`--%throws`, `--%throws(abe, 723pf)`.
251286

252-
```
287+
Example:
288+
```sql
253289
create or replace package example_pgk as
254290

255-
-- %suite(Example Throws Annotation)
291+
--%suite(Example Throws Annotation)
256292

257-
-%test(Throws one of the listed exceptions)
293+
--%test(Throws one of the listed exceptions)
258294
--%throws(-20145,-20146, -20189 ,-20563)
259295
procedure raised_one_listed_exception;
260296

261-
--%test(Throws diff exception)
297+
--%test(Throws different exception than expected)
262298
--%throws(-20144)
263-
procedure raised_diff_exception;
299+
procedure raised_different_exception;
300+
301+
--%test(Throws different exception than listed)
302+
--%throws(-20144,-00001,-20145)
303+
procedure raised_unlisted_exception;
264304

265-
--%test(Givess failure when a exception is expected and nothing is thrown)
305+
--%test(Gives failure when an exception is expected and nothing is thrown)
266306
--%throws(-20459, -20136, -20145)
267307
procedure nothing_thrown;
268308

269309
end;
270-
```
271-
272-
```
273-
create package body annotated_package_with_throws is
310+
/
311+
create or replace package body example_pgk is
274312
procedure raised_one_listed_exception is
275313
begin
276314
raise_application_error(-20189, 'Test error');
277315
end;
278316

279-
procedure raised_diff_exception is
317+
procedure raised_different_exception is
318+
begin
319+
raise_application_error(-20143, 'Test error');
320+
end;
321+
322+
procedure raised_unlisted_exception is
280323
begin
281324
raise_application_error(-20143, 'Test error');
282325
end;
283326

284327
procedure nothing_thrown is
285328
begin
286-
null;
329+
ut.expect(1).to_equal(1);
287330
end;
288331
end;
332+
/
333+
334+
exec ut.run('example_pgk');
289335
```
290336

291-
The test results are:
292-
337+
Running the test will give report:
293338
```
294339
Example Throws Annotation
295-
Throws one of the listed exceptions [.011 sec]
296-
Throws diff exception [.011 sec] (FAILED - 1)
297-
Givess failure when a exception is expected and nothing is thrown [.014 sec] (FAILED - 2)
298-
340+
Throws one of the listed exceptions [.018 sec]
341+
Throws different exception than expected [.008 sec] (FAILED - 1)
342+
Throws different exception than listed [.007 sec] (FAILED - 2)
343+
Gives failure when an exception is expected and nothing is thrown [.002 sec] (FAILED - 3)
344+
299345
Failures:
300-
301-
1) raised_diff_exception
302-
UT3.annotated_package_with_throws.raised_diff_exception Actual: -20143 was expected to be one of (-20144)
303-
304-
2) nothing_thrown
305-
UT3.annotated_package_with_throws.nothing_thrown Expected one of exceptions (-20459,-20136,-20145) but nothing was raised.
306-
307-
Finished in .040591 seconds
308-
3 tests, 2 failed, 0 errored, 0 disabled, 0 warning(s)
346+
347+
1) raised_different_exception
348+
Actual: -20143 was expected to equal: -20144
349+
ORA-20143: Test error
350+
ORA-06512: at "UT3.EXAMPLE_PGK", line 9
351+
ORA-06512: at line 6
352+
353+
2) raised_unlisted_exception
354+
Actual: -20143 was expected to be one of: (-20144, -1, -20145)
355+
ORA-20143: Test error
356+
ORA-06512: at "UT3.EXAMPLE_PGK", line 14
357+
ORA-06512: at line 6
358+
359+
3) nothing_thrown
360+
Expected one of exceptions (-20459, -20136, -20145) but nothing was raised.
361+
362+
Finished in .038692 seconds
363+
4 tests, 3 failed, 0 errored, 0 disabled, 0 warning(s)
309364
```

‎docs/userguide/exception-reporting.md‎

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -2,19 +2,19 @@
22

33
The utPLSQL is responsible for handling exceptions wherever they occur in the test run. utPLSQL is trapping most of the exceptions so that the test execution is not affected by individual tests or test packages throwing an exception.
44
The framework provides a full stacktrace for every exception that was thrown. The stacktrace is clean and does not include any utPLSQL library calls in it.
5-
To achieve rerunability, the ORA-04068, ORA-04061 exceptions are not handled and test execution will be interrupted if such exception is encountered. This is because of how Oracle behaves on those exceptions.
5+
To achieve rerunability, the package state invalidation exceptions (ORA-04068, ORA-04061) are not handled and test execution will be interrupted if such exceptions are encountered. This is because of how Oracle behaves on those exceptions.
66

77
Test execution can fail for different reasons. The failures on different exceptions are handled as follows:
8-
* A test package without body - each `%test` is reported as failed with exception, nothing is executed
9-
* A test package with _invalid body_ - each `%test` is reported as failed with exception, nothing is executed
8+
* A test package without body - each `--%test` is reported as failed with exception, nothing is executed
9+
* A test package with _invalid body_ - each `--%test` is reported as failed with exception, nothing is executed
1010
* A test package with _invalid spec_ - package is not considered a valid unit test package and is excluded from execution. When trying to run a test package with invalid spec explicitly, exception is raised. Only valid specifications are parsed for annotations
11-
* A test package that is raising an exception in `%beforeall` - each `%test` is reported as failed with exception, `%test`, `%beforeeach`, `%beforetest`, `%aftertest` and `%aftereach` are not executed. `%afterall` is executed to allow cleanup of whatever was done in `%beforeall`
12-
* A test package that is raising an exception in `%beforeeach` - each `%test` is reported as failed with exception, `%test`, `%beforetest` and `%aftertest` is not executed. The `%aftereach` and `%afterall` blocks are getting executed to allow cleanup of whatever was done in `%before...` blocks
13-
* A test package that is raising an exception in `%beforetest` - the `%test` is reported as failed with exception, `%test` is not executed. The `%aftertest`, `%aftereach` and `%afterall` blocks are getting executed to allow cleanup of whatever was done in `%before...` blocks
14-
* A test package that is raising an exception in `%test` - the `%test` is reported as failed with exception. The execution of other blocks continues normally
15-
* A test package that is raising an exception in `%aftertest` - the `%test` is reported as failed with exception. The execution of other blocks continues normally
16-
* A test package that is raising an exception in `%aftereach` - each `%test` is reported as failed with exception.
17-
* A test package that is raising an exception in `%afterall` - all blocks of the package are executed, as the `%afterall` is the last step of package execution. Exception in `%afterall` is not affecting test results. A warning with exception stacktrace is displayed in the summary
11+
* A test package that is raising an exception in `--%beforeall` - each `--%test` is reported as failed with exception, `--%test`, `--%beforeeach`, `--%beforetest`, `--%aftertest` and `--%aftereach` are not executed. `--%afterall` is executed to allow cleanup of whatever was done in `--%beforeall`
12+
* A test package that is raising an exception in `--%beforeeach` - each `--%test` is reported as failed with exception, `--%test`, `--%beforetest` and `--%aftertest` is not executed. The `--%aftereach` and `--%afterall` blocks are getting executed to allow cleanup of whatever was done in `--%before...` blocks
13+
* A test package that is raising an exception in `--%beforetest` - the `--%test` is reported as failed with exception, `--%test` is not executed. The `--%aftertest`, `--%aftereach` and `--%afterall` blocks are getting executed to allow cleanup of whatever was done in `--%before...` blocks
14+
* A test package that is raising an exception in `--%test` - the `--%test` is reported as failed with exception. The execution of other blocks continues normally
15+
* A test package that is raising an exception in `--%aftertest` - the `--%test` is reported as failed with exception. The execution of other blocks continues normally
16+
* A test package that is raising an exception in `--%aftereach` - each `--%test` is reported as failed with exception.
17+
* A test package that is raising an exception in `--%afterall` - all blocks of the package are executed, as the `--%afterall` is the last step of package execution. Exception in `--%afterall` is not affecting test results. A warning with exception stacktrace is displayed in the summary
1818

1919

2020
Example of reporting with exception thrown in `%beforetest`:

0 commit comments

Comments
 (0)