How long does the "procedure under test" run usually, outside of a test situation?

How long does the "procedure under test" run usually, outside of a test situation?

Approximately 15-20 minutes

25% overhead doesn't sound incredibly much to me.
As for the coverage: I would assume that the main performance issue is the profiler here, but I'm not the expert on the matter. Maybe one of the others can give some insights.

25% overhead doesn't sound incredibly much to me.
As for the coverage: I would assume that the main performance issue is the profiler here, but I'm not the expert on the matter. Maybe one of the others can give some insights.

I'm Sorry, I overlooked the sentence
Outside of a Test situation, the "Procedure under test" usually takes a minute to execute.

How many lines of code is there in the entire schema?
How many other procedures/packages are called by the tested procedure?
How big is the coverage report (MB)?
How much of coverage report is relevant for you? Do you ou need whole schema coverage or only coverage for rested package(s)?

How many lines of code is there in the entire schema? -- there are 100's of procedures in the schema. So we don't have an exact count on the number of lines of code.
How many other procedures/packages are called by the tested procedure? -- In the Test Procedure we are calling only a single procedure
How big is the coverage report (MB)? -- We have not generated the report, as it was showing execution in progress for more than 2hrs for generating schema level coverage report.
How much of coverage report is relevant for you? Do you need whole schema coverage or only coverage for rested package(s)? -- We need the Coverage to be generated for all the procedures in the main user schema for which the testcases are written in UnitTest schema

Code to be Tested

Here's a small test I've created. There is also a variant with dummy procedures to get a package body with 5000 lines of code. It's attached. However, the execution times are very similar.

Please note that there are no SQL calls in this code. That's intended. The idea is to show the overhead of the profiler when running such code.

create or replace package fibonacci is
  function last_number (a_max_value integer) return integer;
  function total_numbers (a_max_value integer) return integer;
end;
/

create or replace package body fibonacci is
  function last_number (a_max_value in integer) return integer is
    l_current    integer := 0;
    l_prev       integer := 1;
    l_prev_prev  integer := 0;
  begin
    while l_prev_prev + l_prev <= a_max_value loop
      l_current    := l_prev_prev + l_prev;
      l_prev_prev  := l_prev;
      l_prev       := l_current;
    end loop;
    return l_current;
  end last_number;

  function total_numbers (a_max_value integer) return integer is
    l_total integer := 0;
  begin
    for i in 1..a_max_value loop
      if ( last_number(i) = i ) then
        l_total := l_total + 1;
      end if;
    end loop;
    return l_total;
  end total_numbers;

end fibonacci;
/

Test Code

Here's the test code. The first one runs fast (a few millis), the second one is slow (several seconds).

create or replace package test_fibonacci is
  --%suite(test_fibonacci)
  --%suitepath(alltests)

  --%test
  procedure last_number;

   --%test
  procedure total_numbers;
end test_fibonacci;
/

create or replace package body test_fibonacci is
  procedure last_number is
    l_actual    integer;
    l_expected  integer := 832040;
  begin
    l_actual := fibonacci.last_number(1e6);
    ut.expect(l_actual).to_equal(l_expected);
  end last_number;

  procedure total_numbers is
    l_actual    integer;
    l_expected  integer := 29;
  begin
    l_actual := fibonacci.total_numbers(1e6);
    ut.expect(l_actual).to_equal(l_expected);
  end total_numbers;
end test_fibonacci;
/

Run Test in SQLcl

I run the following code in SQLcl

set serveroutput on size unlimited
execute ut.run('test_fibonacci');

and the output was

alltests
  test_fibonacci
    last_number [.01 sec]
    total_numbers [5.437 sec]
 
Finished in 5.454135 seconds
2 tests, 0 failed, 0 errored, 0 disabled, 0 warning(s)

Run Test in SQLDev

I run the same test in SQLDev using the utPLSQL extension. Here's the screenshot of the test runner.

It is slightly slower due to the overhead in the GUI. That's expected.

Run Test in SQLDEv with Code Coverage

Now the same test in SQLDev with Code Coverage. Here's the screenshot of the test runner.

And here's the screenshot of the Code Coverage report shown in the browser.

You see there are a lot of executions reported. It's expected that this additional book keeping takes some time. So the question is it the utPLSQL framework or is it the Oracle Database that is the main contributor?

Run Test in SQLcl with dbms_profiler

You are using an Oracle Database 12.1 hence there is no dbms_plsql_code_coverage package. The utPLSQL framework handles that.

So I ran the following code in SQLcl:

set timing on
set serveroutput on

begin
  dbms_profiler.start_profiler;
  dbms_output.put_line(fibonacci.last_number(1e6));
  dbms_output.put_line(fibonacci.total_numbers(1e6));
  dbms_profiler.stop_profiler();
end;
/

The output was:

832040
29


PL/SQL procedure successfully completed.

Elapsed: 00:00:18.743

That's much faster than in SQLDev... but wait, I used an Oracle Database 19c (19.8) for this test. Hence dbms_plsql_code_coverage was used as well.

Run Test in SQLcl with dbms_profiler and dbms_plsql_code_coverage

Therefore I ran this code:

declare
  l_code_coverage_run integer;
begin
  dbms_profiler.start_profiler;
  l_code_coverage_run := dbms_plsql_code_coverage.start_coverage(systimestamp);
  dbms_output.put_line(fibonacci.last_number(1e6));
  dbms_output.put_line(fibonacci.total_numbers(1e6));
  dbms_plsql_code_coverage.stop_coverage();
  dbms_profiler.stop_profiler();
end;
/

The output was:

832040
29


PL/SQL procedure successfully completed.

Elapsed: 00:00:26.486

Now the runtime is quite similar to the one in SQLDev. In fact this shows that the Oracle Database is responsible for the overhead. There is actually nothing we can do within the utPLSQL framework to improve that.

Conclusion

The overhead of CodeCoverage is in extreme cases with a lot of PL/SQL code around 400 to 500%. For PL/SQL code containing mostly SQL with few iteration in loops the overhead should be much smaller. So it really depends on your code.

@PriyankaKeer
Please upgrade utPLSQL to latest version.
There were performance improvements to coverage reporting done as part of version 3.1.8
That could help with your issues.
See #970

Hope this helps.
Thanks to @PhilippSalvisberg for digging it out.

Thanks a lot @PhilippSalvisberg for debugging the issue. Yes the PLSQL code is little complex from our side , and I completely understand the scenario.
Is there any way in UTPLSQL where we can execute all the tests parallely, which may decrease the overall Test execution time?
Thanks in Advance.

Thanks @jgebal
We are using Latest version of UTPLSQL -3.1.8.546.

@PriyankaKeer
In the original issue you've mentioned you're using utPLSQL 3.1.7.3096
Please check to be 100%

The latest lease is 3.1.10- see releases page for details of changes/fixes/improvements in each release.

Jacek

Yes @jgebal , I was referring to UTPLSQL CLI
utPLSQL-cli 3.1.8-SNAPSHOT.local
utPLSQL-java-api 3.1.8.546

Thanks for the info! Will try to update UTPLSQL Version used in SQL Dev and will try.

Just to be clear. We recommend to install the newest version from here: https://github.com/utPLSQL/utPLSQL/releases .

@PhilippSalvisberg Sure..Will try the latest version. Thanks a lot for the prompt response and suggestions.