Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Commit e023d9e

Browse files
jgebaljgebal
authored
Merge pull request #197 from Pazus/fix-for-invalid-package-v2
Tests were skipped if package was invalid
2 parents ca9dadc + ee89c42 commit e023d9e

30 files changed

Lines changed: 749 additions & 84 deletions

‎docs/userguide/annotations.md‎

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -72,9 +72,20 @@ end test_pkg;
7272
| `%aftereach` | Procedure | Denotes that the annotated procedure should be executed after each `%test` method in the current suite. |
7373
| `%beforetest(<procedure_name>)` | Procedure | Denotes that mentioned procedure should be executed before the annotated `%test` procedure. |
7474
| `%aftertest(<procedure_name>)` | Procedure | Denotes that mentioned procedure should be executed after the annotated `%test` procedure. |
75-
| `%rollback(<type>)` | Package/procedure | Configure transaction control behaviour (type). Supported values: `auto`(default) - rollback to savepoint (before the test/suite setup) is issued after each test/suite teardown; `manual` - rollback is never issued automatically. Property can be overridden for child element (test in suite) |
75+
| `%rollback(<type>)` | Package/procedure | Configure transaction control behaviour (type). Supported values: `auto`(default) - A savepoint is created before invocation of each "before block" is and a rollback to specific savepoint is issued after each "after" block; `manual` - rollback is never issued automatically. Property can be overridden for child element (test in suite) |
7676
| `%disabled` | Package/procedure | Used to disable a suite or a test |
7777

78+
# Using automatic rollbacks in tests
79+
By default, every test is isolated from other tests using savepoint.
80+
This solution is suitable for use-cases, where the code that is getting tested as well as the unit tests themselves do not use transaction control commands (commit/rollback).
81+
In general, your unit tests should not use transaction control as long as the core you are testing is not using it too.
82+
Keeping the transactions uncommitted allows your changes to be isolated and the execution of tests is not impacting others that might be using a shared (integration) development database.
83+
84+
If however you're in situation, where the code you are testing, is using transaction control (like ETL code is usually doing), then your tests should not use the default rollback(auto)
85+
You should make sure that thr entire suitepath all the way to the root is using manual transaction control in that case.
86+
87+
In some cases it is needed to perform DDL in setup/teardown. It is recommended to move such DDL statements to a procedure with pragma autonomous_transaction to eliminate implicit commit of the main session.
88+
7889
# Suitepath concept
7990
It is very likely that the application for which you are going to introduce tests consists of many different packages or procedures/functions. Usually procedures can be logically grouped inside a package, there also might be several logical groups of procedure in a single package or even packages themselves might relate to a common module.
8091

@@ -148,3 +159,4 @@ A `%suitepath` can be provided in tree ways:
148159
* schema - execute all test in the schema
149160
* [schema]:suite1[.suite2][.suite3]...[.procedure] - execute all tests in all suites from suite1[.suite2][.suite3]...[.procedure] path. If schema is not provided, then current schema is used. Example: `:all.rooms_tests`.
150161
* [schema.]package[.procedure] - execute all tests in the test package provided. The whole hierarchy of suites in the schema is build before, all before/after hooks of partn suites for th provided suite package are executed as well. Example: `tests.test_contact.test_last_name_validator` or simply `test_contact.test_last_name_validator` if `tests` is the current schema.
162+

‎docs/userguide/exception-reporting.md‎

Lines changed: 121 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,121 @@
1+
# Exception handling and reporting
2+
3+
The utPLSQL is responsible for handling exceptions wherever they occur in the test run. utPLSQL is trapping most of the exceptions so that the test execution is not affected by individual tests or test packages throwing an exception.
4+
The framework provides a full stacktrace for every exception that was thrown. The stacktrace is clean and does not include any utPLSQL library calls in it.
5+
To achieve rerunability, the ORA-04068, ORA-04061 exceptions are not handled and test execution will be interrupted if such exception is encountered. This is because of how Oracle behaves on those exceptions.
6+
7+
Test execution can fail for different reasons. The failures on different exceptions are handled as follows:
8+
* A test package without body - each `%test` is reported as failed with exception, nothing is executed
9+
* A test package with _invalid body_ - each `%test` is reported as failed with exception, nothing is executed
10+
* A test package with _invalid spec_ - package is not considered a valid unit test package and is excluded from execution. When trying to run a test package with invalid spec explicitly, exception is raised. Only valid specifications are parsed for annotations
11+
* A test package that is raising an exception in `%beforeall` - each `%test` is reported as failed with exception, `%test`, `%beforeeach`, `%beforetest`, `%aftertest` and `%aftereach` are not executed. `%afterall` is executed to allow cleanup of whatever was done in `%beforeall`
12+
* A test package that is raising an exception in `%beforeeach` - each `%test` is reported as failed with exception, `%test`, `%beforetest` and `%aftertest` is not executed. The `%aftereach` and `%afterall` blocks are getting executed to allow cleanup of whatever was done in `%before...` blocks
13+
* A test package that is raising an exception in `%beforetest` - the `%test` is reported as failed with exception, `%test` is not executed. The `%aftertest`, `%aftereach` and `%afterall` blocks are getting executed to allow cleanup of whatever was done in `%before...` blocks
14+
* A test package that is raising an exception in `%test` - the `%test` is reported as failed with exception. The execution of other blocks continues normally
15+
* A test package that is raising an exception in `%aftertest` - the `%test` is reported as failed with exception. The execution of other blocks continues normally
16+
* A test package that is raising an exception in `%aftereach` - all blocks of the package are executed, as ehe `%aftereach` is a closing block for an individual test. Exception in `%aftereach` is not affecting test results. For every failed execution of `%aftereach` a warning with exception stacktrace is displayed in the summary
17+
* A test package that is raising an exception in `%afterall` - all blocks of the package are executed, as the `%afterall` is the last step of package execution. Exception in `%afterall` is not affecting test results. A warning with exception stacktrace is displayed in the summary
18+
19+
20+
Example of reporting with exception thrown in `%beforetest`:
21+
````
22+
Remove rooms by name
23+
Removes a room without content in it (FAILED - 1)
24+
Does not remove room when it has content
25+
Raises exception when null room name given
26+
27+
Failures:
28+
29+
1) remove_empty_room
30+
31+
error: ORA-20001: Test exception
32+
ORA-06512: at "UT3.TEST_REMOVE_ROOMS_BY_NAME", line 39
33+
ORA-06512: at line 6
34+
35+
Finished in ,039346 seconds
36+
3 tests, 0 failed, 1 errored, 0 ignored.
37+
````
38+
39+
Example of reporting with exception thrown in `%test`:
40+
```
41+
Remove rooms by name
42+
Removes a room without content in it (FAILED - 1)
43+
Does not remove room when it has content
44+
Raises exception when null room name given
45+
46+
Failures:
47+
48+
1) remove_empty_room
49+
50+
error: ORA-20001: Test exception
51+
ORA-06512: at "UT3.TEST_REMOVE_ROOMS_BY_NAME", line 48
52+
ORA-06512: at line 6
53+
54+
Finished in ,035726 seconds
55+
3 tests, 0 failed, 1 errored, 0 ignored.
56+
```
57+
58+
Example of reporting with exception thrown in `%aftertest`:
59+
```
60+
Remove rooms by name
61+
Removes a room without content in it (FAILED - 1)
62+
Does not remove room when it has content
63+
Raises exception when null room name given
64+
65+
Failures:
66+
67+
1) remove_empty_room
68+
69+
error: ORA-20001: Test exception
70+
ORA-06512: at "UT3.TEST_REMOVE_ROOMS_BY_NAME", line 42
71+
ORA-06512: at line 6
72+
73+
Finished in ,045523 seconds
74+
3 tests, 0 failed, 1 errored, 0 ignored.
75+
```
76+
77+
Example of reporting with exception thrown in `%aftereach`:
78+
```
79+
Remove rooms by name
80+
Removes a room without content in it
81+
Does not remove room when it has content
82+
Raises exception when null room name given
83+
84+
Warnings:
85+
86+
1) test_remove_rooms_by_name - Aftereach procedure failed:
87+
ORA-20001: Test exception
88+
ORA-06512: at "UT3.TEST_REMOVE_ROOMS_BY_NAME", line 31
89+
ORA-06512: at line 6
90+
91+
2) test_remove_rooms_by_name - Aftereach procedure failed:
92+
ORA-20001: Test exception
93+
ORA-06512: at "UT3.TEST_REMOVE_ROOMS_BY_NAME", line 31
94+
ORA-06512: at line 6
95+
96+
3) test_remove_rooms_by_name - Aftereach procedure failed:
97+
ORA-20001: Test exception
98+
ORA-06512: at "UT3.TEST_REMOVE_ROOMS_BY_NAME", line 31
99+
ORA-06512: at line 6
100+
101+
Finished in ,05071 seconds
102+
3 tests, 0 failed, 0 errored, 0 ignored. 3 warning(s)
103+
```
104+
105+
Example of reporting with exception thrown in `%afterall`:
106+
```
107+
Remove rooms by name
108+
Removes a room without content in it
109+
Does not remove room when it has content
110+
Raises exception when null room name given
111+
112+
Warnings:
113+
114+
1) test_remove_rooms_by_name - Afterall procedure failed:
115+
ORA-20001: Test exception
116+
ORA-06512: at "UT3.TEST_REMOVE_ROOMS_BY_NAME", line 35
117+
ORA-06512: at line 6
118+
119+
Finished in ,044902 seconds
120+
3 tests, 0 failed, 0 errored, 0 ignored. 1 warning(s)
121+
```

‎source/core/types/ut_event_listener.tpb‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -88,7 +88,7 @@ create or replace type body ut_event_listener is
8888
end if;
8989
end loop;
9090

91-
end;
91+
end fire_event;
9292

9393
end;
9494
/

‎source/core/types/ut_logical_suite.tpb‎

Lines changed: 21 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -52,39 +52,35 @@ create or replace type body ut_logical_suite as
5252
self.items(self.items.last) := a_item;
5353
end;
5454

55-
overriding member procedure do_execute(self in out nocopy ut_logical_suite, a_listener in out nocopy ut_event_listener_base) is
56-
l_completed_without_errors boolean;
57-
begin
58-
l_completed_without_errors := self.do_execute(a_listener);
59-
end;
60-
6155
overriding member function do_execute(self in out nocopy ut_logical_suite, a_listener in out nocopy ut_event_listener_base) return boolean is
6256
l_suite_savepoint varchar2(30);
6357
l_item_savepoint varchar2(30);
6458
l_completed_without_errors boolean;
6559
begin
6660
ut_utils.debug_log('ut_logical_suite.execute');
61+
62+
a_listener.fire_before_event(ut_utils.gc_suite,self);
63+
self.start_time := current_timestamp;
6764

6865
if self.get_ignore_flag() then
6966
self.result := ut_utils.tr_ignore;
67+
self.end_time := self.start_time;
7068
ut_utils.debug_log('ut_logical_suite.execute - ignored');
7169
else
72-
a_listener.fire_before_event(ut_utils.gc_suite,self);
7370

7471
self.start_time := current_timestamp;
7572

7673
for i in 1 .. self.items.count loop
7774
-- execute the item (test or suite)
7875
self.items(i).do_execute(a_listener);
79-
8076
end loop;
8177

8278
self.calc_execution_result();
83-
8479
self.end_time := current_timestamp;
8580

86-
a_listener.fire_after_event(ut_utils.gc_suite,self);
8781
end if;
82+
83+
a_listener.fire_after_event(ut_utils.gc_suite,self);
8884

8985
return l_completed_without_errors;
9086
end;
@@ -103,7 +99,21 @@ create or replace type body ut_logical_suite as
10399
end if;
104100

105101
self.result := l_result;
106-
end;
102+
end;
103+
104+
overriding member procedure fail(self in out nocopy ut_logical_suite, a_listener in out nocopy ut_event_listener_base, a_failure_msg varchar2) is
105+
begin
106+
ut_utils.debug_log('ut_logical_suite.fail');
107+
a_listener.fire_before_event(ut_utils.gc_suite, self);
108+
self.start_time := current_timestamp;
109+
for i in 1 .. self.items.count loop
110+
-- execute the item (test or suite)
111+
self.items(i).fail(a_listener,a_failure_msg);
112+
end loop;
113+
self.calc_execution_result();
114+
self.end_time := self.start_time;
115+
a_listener.fire_after_event(ut_utils.gc_suite, self);
116+
end;
107117

108118
end;
109119
/

‎source/core/types/ut_logical_suite.tps‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
create or replace type ut_logical_suite force under ut_suite_item (
1+
create or replace type ut_logical_suite under ut_suite_item (
22
/*
33
utPLSQL - Version X.X.X.X
44
Copyright 2016 - 2017 utPLSQL Project
@@ -31,7 +31,7 @@ create or replace type ut_logical_suite force under ut_suite_item (
3131
member function item_index(a_name varchar2) return pls_integer,
3232
member procedure add_item(self in out nocopy ut_logical_suite, a_item ut_suite_item),
3333
overriding member function do_execute(self in out nocopy ut_logical_suite, a_listener in out nocopy ut_event_listener_base) return boolean,
34-
overriding member procedure do_execute(self in out nocopy ut_logical_suite, a_listener in out nocopy ut_event_listener_base),
35-
overriding member procedure calc_execution_result(self in out nocopy ut_logical_suite)
34+
overriding member procedure calc_execution_result(self in out nocopy ut_logical_suite),
35+
overriding member procedure fail(self in out nocopy ut_logical_suite, a_listener in out nocopy ut_event_listener_base, a_failure_msg varchar2)
3636
) not final
3737
/

‎source/core/types/ut_reporter_base.tpb‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -131,5 +131,6 @@ create or replace type body ut_reporter_base is
131131
begin
132132
ut_output_buffer.close(self);
133133
end;
134+
134135
end;
135136
/

‎source/core/types/ut_reporter_base.tps‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
create or replace type ut_reporter_base force authid current_user as object(
1+
create or replace type ut_reporter_base authid current_user as object(
22
/*
33
utPLSQL - Version X.X.X.X
44
Copyright 2016 - 2017 utPLSQL Project

‎source/core/types/ut_results_counter.tpb‎

Lines changed: 12 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -17,10 +17,11 @@ create or replace type body ut_results_counter as
1717
*/
1818
constructor function ut_results_counter(self in out nocopy ut_results_counter) return self as result is
1919
begin
20-
self.ignored_count := 0;
21-
self.success_count := 0;
22-
self.failure_count := 0;
23-
self.errored_count := 0;
20+
self.ignored_count := 0;
21+
self.success_count := 0;
22+
self.failure_count := 0;
23+
self.errored_count := 0;
24+
self.warnings_count := 0;
2425
return;
2526
end;
2627

@@ -39,10 +40,17 @@ create or replace type body ut_results_counter as
3940
self.success_count := self.success_count + a_item.success_count;
4041
self.failure_count := self.failure_count + a_item.failure_count;
4142
self.errored_count := self.errored_count + a_item.errored_count;
43+
self.warnings_count := self.warnings_count + a_item.warnings_count;
44+
end;
45+
46+
member procedure increase_warning_count(self in out nocopy ut_results_counter) is
47+
begin
48+
self.warnings_count := self.warnings_count + 1;
4249
end;
4350

4451
member function total_count return integer is
4552
begin
53+
--skip warnings here
4654
return self.ignored_count + self.success_count + self.failure_count + self.errored_count;
4755
end;
4856

‎source/core/types/ut_results_counter.tps‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,9 +19,11 @@ create or replace type ut_results_counter as object(
1919
success_count integer,
2020
failure_count integer,
2121
errored_count integer,
22+
warnings_count integer,
2223
constructor function ut_results_counter(self in out nocopy ut_results_counter) return self as result,
2324
constructor function ut_results_counter(self in out nocopy ut_results_counter, a_status integer) return self as result,
2425
member procedure sum_counter_values(self in out nocopy ut_results_counter, a_item ut_results_counter),
26+
member procedure increase_warning_count(self in out nocopy ut_results_counter),
2527
member function total_count return integer,
2628
member function result_status return integer
2729
)

‎source/core/types/ut_run.tpb‎

Lines changed: 20 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -24,12 +24,6 @@ create or replace type body ut_run as
2424
return;
2525
end;
2626

27-
overriding member procedure do_execute(self in out nocopy ut_run, a_listener in out nocopy ut_event_listener_base) is
28-
l_completed_without_errors boolean;
29-
begin
30-
l_completed_without_errors := self.do_execute(a_listener);
31-
end;
32-
3327
overriding member function do_execute(self in out nocopy ut_run, a_listener in out nocopy ut_event_listener_base) return boolean is
3428
l_completed_without_errors boolean;
3529
begin
@@ -38,6 +32,9 @@ create or replace type body ut_run as
3832
a_listener.fire_before_event(ut_utils.gc_run, self);
3933

4034
self.start_time := current_timestamp;
35+
36+
-- clear anything that might stay in the session's cache
37+
ut_assert_processor.clear_asserts;
4138

4239
for i in 1 .. self.items.count loop
4340
l_completed_without_errors := self.items(i).do_execute(a_listener);
@@ -67,6 +64,23 @@ create or replace type body ut_run as
6764

6865
self.result := l_result;
6966
end;
67+
68+
overriding member procedure fail(self in out nocopy ut_run, a_listener in out nocopy ut_event_listener_base, a_failure_msg varchar2) is
69+
begin
70+
ut_utils.debug_log('ut_run.fail');
71+
72+
a_listener.fire_before_event(ut_utils.gc_run, self);
73+
self.start_time := current_timestamp;
74+
75+
for i in 1 .. self.items.count loop
76+
self.items(i).fail(a_listener, a_failure_msg);
77+
end loop;
78+
79+
self.calc_execution_result();
80+
self.end_time := self.start_time;
81+
82+
a_listener.fire_after_event(ut_utils.gc_run, self);
83+
end;
7084

7185
end;
7286
/

0 commit comments

Comments
 (0)