Closed
Description
Current state
When building reusable expectation procedures, the failure message doesn't provide sufficient in formation for identifying the lines where expectations have failures.
So if I have few tests using the same procedure and each test has more than one expectation, it's really hard or maybe impossible to track which one has failed.
Example:
create or replace package check_stack is
--%suite
--%test
procedure test_1;
--%test
procedure test_2;
end;
/
create or replace package body check_stack is
procedure do_check( p_actual varchar2, p_matcher ut_matcher ) is
begin
ut.expect(p_actual).to_( p_matcher );
end;
function get_actual return varchar2 is
begin
return chr(ascii('a') + trunc(dbms_random.value(0,3)));
end;
procedure test_1 is
begin
do_check( get_actual(), equal('b'));
do_check( get_actual(), equal('b'));
do_check( get_actual(), equal('b'));
end;
procedure test_2 is
begin
do_check('ab'||get_actual()||'ef', be_like('%c%'));
do_check('ab'||get_actual()||'ef', be_like('%c%'));
do_check('ab'||get_actual()||'ef', be_like('%c%'));
end;
end;
/
exec ut.run('check_stack');Can produce outputs as below:
check_stack
test_1 [.005 sec] (FAILED - 1)
test_2 [.008 sec] (FAILED - 2)
Failures:
1) test_1
Actual: 'c' (varchar2) was expected to equal: 'b' (varchar2)
at "UT3.CHECK_STACK.DO_CHECK", line 5 ut.expect(p_actual).to_( p_matcher );
2) test_2
Actual: 'abaef' (varchar2) was expected to be like: '%c%'
at "UT3.CHECK_STACK.DO_CHECK", line 5 ut.expect(p_actual).to_( p_matcher );
Actual: 'abaef' (varchar2) was expected to be like: '%c%'
at "UT3.CHECK_STACK.DO_CHECK", line 5 ut.expect(p_actual).to_( p_matcher );
Finished in .017699 seconds
2 tests, 2 failed, 0 errored, 0 disabled, 0 warning(s)
Proposed solution
On failures, show full stack of calls within a test so that the outputs from below failures could look like this:
check_stack
test_1 [.005 sec] (FAILED - 1)
test_2 [.008 sec] (FAILED - 2)
Failures:
1) test_1
Actual: 'c' (varchar2) was expected to equal: 'b' (varchar2)
at "UT3.CHECK_STACK.DO_CHECK", line 5 ut.expect(p_actual).to_( p_matcher );
at "UT3.CHECK_STACK.DO_CHECK", line 16
2) test_2
Actual: 'abaef' (varchar2) was expected to be like: '%c%'
at "UT3.CHECK_STACK.DO_CHECK", line 5 ut.expect(p_actual).to_( p_matcher );
at "UT3.CHECK_STACK.DO_CHECK", line 22
Actual: 'abaef' (varchar2) was expected to be like: '%c%'
at "UT3.CHECK_STACK.DO_CHECK", line 5 ut.expect(p_actual).to_( p_matcher );
at "UT3.CHECK_STACK.DO_CHECK", line 24
Finished in .017699 seconds
2 tests, 2 failed, 0 errored, 0 disabled, 0 warning(s)
With such addition, it will be much easier to navigate to the rig place in test code and identify the root cause of failure.
@PhilippSalvisberg - would that impact SQLDeveloper plugin work you're doing? Would there be any additional rework needed on your end?