@TobiasDo1 This is an interesting case.

Does the code coverage run contain a test case that passes an identical value for p_short_desc AND a different value for p_description (l_description must be NOT NULL)? IMO this is is required to get full coverage of this code section.

So my assumption is, that your tests in fact do not cover this code section and the result is correct.

Hello Philipp,

thanks for your quick reply.

Well, indeed I think that this one line is not being covered, but I would have expected that only line 338 is being marked red in the html output.

I did run the identical test a 2nd time and now the html shows lines 337 and 338 to be uncovered. According to dbms_utplsql_code_coverage line 337 should be covered. That is what looks weird to me.

Best regards
Tobias

@TobiasDo1 I was too quick with my assumption. Sorry about that. When run_id 14881 corresponds to the HTML output than this looks wrong.

Can you please provide the version information as described in https://github.com/utPLSQL/utPLSQL/issues/new?assignees=&labels=&template=bug_report.md&title=? Thank you.

Sure, here they are.
DB Version:
19.0.0.0.0
19.7.0

ut_version:
v3.1.10.3349

banner:
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.7.0.0.0

nls_session_parameters:
NLS_LANGUAGE AMERICAN
NLS_TERRITORY AMERICA
NLS_CURRENCY $
NLS_ISO_CURRENCY AMERICA
NLS_NUMERIC_CHARACTERS .,
NLS_CALENDAR GREGORIAN
NLS_DATE_FORMAT DD-Mon-RRRR
NLS_DATE_LANGUAGE AMERICAN
NLS_SORT BINARY
NLS_TIME_FORMAT HH.MI.SSXFF AM
NLS_TIMESTAMP_FORMAT DD-MON-RR HH.MI.SSXFF AM
NLS_TIME_TZ_FORMAT HH.MI.SSXFF AM TZR
NLS_TIMESTAMP_TZ_FORMAT DD-MON-RR HH.MI.SSXFF AM TZR
NLS_DUAL_CURRENCY $
NLS_COMP BINARY
NLS_LENGTH_SEMANTICS BYTE
NLS_NCHAR_CONV_EXCP FALSE

port_string:
IBMPC/WIN_NT64-9.1.0

Thanks @TobiasDo1. That looks good. Now we need to make this reproducible in our environment.

@TobiasDo1: How exactly do you reproduce the HTML Report?

I am running code coverage option out of the context menu from the sql developer plugin.

And checking sys coverage via:

SELECT LISTAGG(ccb.col,',') WITHIN GROUP (ORDER BY ccb.col) AS col,
LISTAGG(ccb.covered,',') WITHIN GROUP (ORDER BY ccb.col) AS covered,
ccu.run_id,
s.line,
s.text
FROM user_source s JOIN dbmspcc_units ccu ON s.name = ccu.name AND s.TYPE = ccu.TYPE
LEFT OUTER JOIN dbmspcc_blocks ccb ON ccu.run_id = ccb.run_id AND ccu.object_id = ccb.object_id AND s.line = ccb.line
WHERE LOWER(s.name) = 'proc_art'
AND ccu.run_id = 14941
AND s.line BETWEEN 331 AND 345
GROUP BY ccu.run_id,
s.line,
s.text
ORDER BY 3 DESC,
4;

where run_id is latest.

Thanks. I try to reproduce it over the weekend.

I tried to reproduce it with the following function under test:

create or replace function f (
   a1  in  varchar2,
   a2  in  varchar2,
   b1  in  varchar2,
   b2  in  varchar2
) return integer is
begin
   if a1 is not null and a2 is null
      or a1 is null and a2 is not null 
      or a1 != a2
      or b1 is not null and b2 is null
      or b1 is null and b2 is not null
      or b1 != b2 
   then
      return 1;
   end if;
   return 0;
end;
/

I created the package test_f to test the function f:

create or replace package test_f is
   --%suite

   --%test
   procedure p_null_null_null_null;

   --%test
   procedure p_a_a_b_b;

   --%test
   procedure p_a1_a2_null_null;
end;
/

create or replace package body test_f is
   procedure p_null_null_null_null is
   begin
      ut.expect(f(null, null, null, null)).to_equal(0);
   end;

   procedure p_a_a_b_b is
   begin
      ut.expect(f('a', 'a', 'b', 'b')).to_equal(0);
   end;

   procedure p_a1_a2_null_null is
   begin
      ut.expect(f('a1', 'a2', null, null)).to_equal(1);
   end;
end;
/

I ran all tests from SQLDeveloper connected as user plscope against an 21c database with the newest utPLSQL version installed in schema ut3.

When I run the complete package test_f, the code coverage result is:

When I run only test_f.p_null_null_null_null, the code coverage result is:

When I run only test_f.p_a_a_b_b, the code coverage result is:

When I run only test_f.p_a1_a2_null_null, the code coverage result is:

For me the results are plausible (beside the counters when running the complete package, but I do not consider this a utPLSQL issue). utPLSQL combines the result from the PL/SQL Profiler and the results from the 12.2 Code Block Coverage. The tables are installed in the utPLSQL schema. So, to validate if HTML report for code coverage is valid, you have to query the right tables. For example for the last run this would be something like this:

with
   obj as (
      select * 
        from dba_objects
       where owner = 'PLSCOPE'
         and object_name = 'F'
         and object_type = 'FUNCTION'
   ),
   source as (
      select s.*
        from dba_source s
        join obj
          on obj.owner = s.owner
             and obj.object_type = s.type
             and obj.object_name = s.name
   ),
   coverage_runs as (
      select *
        from ut3.ut_coverage_runs
       where block_coverage_id = (
               select max(block_coverage_id)
                 from ut3.ut_coverage_runs
             )                              
   ),
   line_coverage as (
      select d.line#,
             case
                when sum(d.total_occur) = 0 and sum(d.total_time) > 0 then
                   1
                else
                   sum(d.total_occur)
             end line_coverage_count
        from obj
        join ut3.plsql_profiler_units u
          on u.unit_owner = obj.owner
             and u.unit_type = obj.object_type
             and u.unit_name = obj.object_name
        join ut3.plsql_profiler_data d
          on u.runid = d.runid
         and u.unit_number = d.unit_number
        join coverage_runs r
          on r.line_coverage_id = u.runid
       group by d.line#
   ),
   block_coverage as (
      select line as line,
             count(block) as blocks,
             sum(covered) as covered_blocks
        from (
                select ccb.line,
                       ccb.block,
                       max(ccb.covered) as covered
                  from obj 
                  join ut3.dbmspcc_units ccu
                    on ccu.object_id = obj.object_id
                  join coverage_runs r
                    on r.block_coverage_id = ccu.run_id
                  left join ut3.dbmspcc_blocks ccb
                    on ccu.run_id = ccb.run_id
                   and ccu.object_id = ccb.object_id
                 group by ccb.line, ccb.block
             )
       group by line
      having count(block) > 1
   )
select s.line, s.text, lc.line_coverage_count, bc.blocks, bc.covered_blocks
  from source s
  left join line_coverage lc
    on s.line = lc.line#
  left join block_coverage bc
    on s.line = bc.line
 order by s.line;

I ran that query as a DBA and the result was:

So this should explain on what data the HTML code coverage report is based. Primary on line coverage. For lines containing more than one block also block coverage is considered. There is some logic to identify relevant lines in the utPLSQL code, which this query does not reflect. It's important to handle Oracle version specific issues and identify what is 100%. However, the query should provide all pieces relevant for a single line of code.

IMO the HTML Code Coverage report is correct and it correctly reflects the data gathered by the PL/SQL Profiler and the 12.2 Code Block Coverage.

I consider this issue more a question regarding the relationship of 12.2 Code Block Coverage and the HTML Code Coverage report.

Good morning Philipp,
maybe I am missing something, but where are objects ut3.ut_coverage_runs and coverage_runs located?

where are objects ut3.ut_coverage_runs and coverage_runs located?

In the schema where utPLSQL v3 is installed. The default is ut3.

I assumed that, but cannot find it.
These are the results from the object search.

Am I missing something from installation?

Hmm, when you've installed utPLSQL 3.1.10 in an Oracle Database 19.7, then the table ut_coverage_runs must exists.

The script @@install_component.sql 'core/coverage/ut_coverage_runs.sql' is called during installation. Here's a screenshot from SQLDev:

And coverage_runs is not a table. It is a named query in the WITH clause.

Hello Philipp,

I tried to reinstall the utplsql framework once again on one of our DBs.

Here is what I did:

These are the log files.
uninstall.log
install.log

But still no ut_coverage_runs table

For installation I took latest 3.1.10 utplsql.zip from Downloads section from github (2020-02-23)

Right, This table was added in this PR and currently available only on the develop branch.

I will install the 3.1.10 in an environment and amend the query for this version.

In utPLSQL 3.1.10 the block coverage tables are all global temporary tables (it is different in the develop branch and will change with 3.1.11). You have to create the block coverage table with the API in the user running the tests:

begin
   dbms_plsql_code_coverage.create_coverage_tables(true);
end;
/

I've done that as user plscope.

Then you can run the following query as a DBA user (change plscope and object filter criteria to match your environment):

with
   obj as (
      select * 
        from dba_objects
       where owner = 'PLSCOPE'
         and object_name = 'F'
         and object_type = 'FUNCTION'
   ),
   source as (
      select s.*
        from dba_source s
        join obj
          on obj.owner = s.owner
             and obj.object_type = s.type
             and obj.object_name = s.name
   ),
   line_coverage as (
      select d.line#,
             case
                when sum(d.total_occur) = 0 and sum(d.total_time) > 0 then
                   1
                else
                   sum(d.total_occur)
             end line_coverage_count
        from obj
        join ut3.plsql_profiler_units u
          on u.unit_owner = obj.owner
             and u.unit_type = obj.object_type
             and u.unit_name = obj.object_name
        join ut3.plsql_profiler_data d
          on u.runid = d.runid
             and u.unit_number = d.unit_number
       where u.runid in (
                select max(r.runid)
                  from ut3.plsql_profiler_runs r
             )
       group by d.line#
   ),
   block_coverage as (
      select line as line,
             count(block) as blocks,
             sum(covered) as covered_blocks
        from (
                select ccb.line,
                       ccb.block,
                       max(ccb.covered) as covered
                  from obj 
                  join plscope.dbmspcc_units ccu
                    on ccu.object_id = obj.object_id
                  left join plscope.dbmspcc_blocks ccb
                    on ccu.run_id = ccb.run_id
                       and ccu.object_id = ccb.object_id
                 where ccu.run_id in (
                          select max(r.run_id)
                            from plscope.dbmspcc_runs r
                       )
                 group by ccb.line, ccb.block
             )
       group by line
      having count(block) > 1
   )
select s.line, s.text, lc.line_coverage_count, bc.blocks, bc.covered_blocks
  from source s
  left join line_coverage lc
    on s.line = lc.line#
  left join block_coverage bc
    on s.line = bc.line
 order by s.line;

The code coverage reports and query results in v3.1.10 and the current develop branch are the same.

Hi Philipp,
just wanted to give you a feedback from the testing so far.

I did execute dbms_plsql_code_coverage.create_coverage_tables(true);
and changed the query above according my filter criterias.

This does work now. I can query the results.

As next step I have opened SQL Developer and run Code Coverage from context menu for one package.
Once completed, I queried the code coverage with code snippet above (filtered on those line items I pay attention to).
This is the output:

Then once again triggering the code coverage and checking query like before:

And once again same tests with different results:

Not sure, if there is something within this test that makes the measurement vary or if if it is something related to test order or anything else.

Thank you for the update. This issue was regarding the deviation of the code coverage tables and the HTML report. The HTML report of these three runs were different (line coverage is leading), right? - In this case I'd like to close this issue.

I cannot explain why the line code coverage is different between the runs. If the test case is not flaky then the only explanation would be a bug in the PL/SQL Profiler.

In your situation I would set a breakpoint at line 327 and inspect the values of p_short_desc, l_short_desc, p_description and l_description. Based on the line coverage you should touch this line only once. I expect that the values differ per test run, but this is just a gut feeling.

Thank you.

Hi Philipp,
thanks a lot for your help.
I am fine to close this issue and we will check the test from our end.