Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Provide more stack info on failed expectations #967

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
jgebal opened this issue Jul 4, 2019 · 0 comments · Fixed by #963
Closed

Provide more stack info on failed expectations #967

jgebal opened this issue Jul 4, 2019 · 0 comments · Fixed by #963
Assignees
@jgebal
Labels
enhancement

Comments

@jgebal
Copy link
Member

jgebal commented Jul 4, 2019

Current state

When building reusable expectation procedures, the failure message doesn't provide sufficient in formation for identifying the lines where expectations have failures.

So if I have few tests using the same procedure and each test has more than one expectation, it's really hard or maybe impossible to track which one has failed.

Example:

create or replace package check_stack is
  --%suite

  --%test
  procedure test_1;
  
  --%test
  procedure test_2;
  
end;
/

create or replace package body check_stack is

  procedure do_check( p_actual varchar2, p_matcher ut_matcher ) is 
  begin
    ut.expect(p_actual).to_( p_matcher );
  end;
  
  function get_actual return varchar2 is
  begin
    return chr(ascii('a') + trunc(dbms_random.value(0,3)));
  end;

  procedure test_1 is
  begin
    do_check( get_actual(), equal('b'));
    do_check( get_actual(), equal('b'));
    do_check( get_actual(), equal('b'));
  end;  

  procedure test_2 is
  begin
    do_check('ab'||get_actual()||'ef', be_like('%c%'));
    do_check('ab'||get_actual()||'ef', be_like('%c%'));
    do_check('ab'||get_actual()||'ef', be_like('%c%'));
  end;

end;
/

exec ut.run('check_stack');

Can produce outputs as below:

check_stack
  test_1 [.005 sec] (FAILED - 1)
  test_2 [.008 sec] (FAILED - 2)
 
Failures:
 
  1) test_1
      Actual: 'c' (varchar2) was expected to equal: 'b' (varchar2)
      at "UT3.CHECK_STACK.DO_CHECK", line 5 ut.expect(p_actual).to_( p_matcher );
      
       
  2) test_2
      Actual: 'abaef' (varchar2) was expected to be like: '%c%' 
      at "UT3.CHECK_STACK.DO_CHECK", line 5 ut.expect(p_actual).to_( p_matcher );
      
       
      Actual: 'abaef' (varchar2) was expected to be like: '%c%' 
      at "UT3.CHECK_STACK.DO_CHECK", line 5 ut.expect(p_actual).to_( p_matcher );
      
       
Finished in .017699 seconds
2 tests, 2 failed, 0 errored, 0 disabled, 0 warning(s)

Proposed solution

On failures, show full stack of calls within a test so that the outputs from below failures could look like this:

check_stack
  test_1 [.005 sec] (FAILED - 1)
  test_2 [.008 sec] (FAILED - 2)
 
Failures:
 
  1) test_1
      Actual: 'c' (varchar2) was expected to equal: 'b' (varchar2)
      at "UT3.CHECK_STACK.DO_CHECK", line 5 ut.expect(p_actual).to_( p_matcher );
      at "UT3.CHECK_STACK.DO_CHECK", line 16
      
       
  2) test_2
      Actual: 'abaef' (varchar2) was expected to be like: '%c%' 
      at "UT3.CHECK_STACK.DO_CHECK", line 5 ut.expect(p_actual).to_( p_matcher );
      at "UT3.CHECK_STACK.DO_CHECK", line 22
      
       
      Actual: 'abaef' (varchar2) was expected to be like: '%c%' 
      at "UT3.CHECK_STACK.DO_CHECK", line 5 ut.expect(p_actual).to_( p_matcher );
      at "UT3.CHECK_STACK.DO_CHECK", line 24
      
       
Finished in .017699 seconds
2 tests, 2 failed, 0 errored, 0 disabled, 0 warning(s)

With such addition, it will be much easier to navigate to the rig place in test code and identify the root cause of failure.

@PhilippSalvisberg - would that impact SQLDeveloper plugin work you're doing? Would there be any additional rework needed on your end?
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jgebal jgebal

Labels
enhancement
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Ability to run expectations without test utPLSQL/utPLSQL
1 participant
@jgebal