From ebb59383d90e64a1b265fe85a7a0034f18ac9d29 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" Date: Sun, 11 Dec 2022 19:27:38 +0000 Subject: [PATCH 01/77] Updated project version after build [skip ci] --- docs/about/authors.md | 2 +- docs/about/license.md | 2 +- docs/about/project-details.md | 2 +- docs/about/support.md | 2 +- docs/index.md | 2 +- docs/userguide/advanced_data_comparison.md | 2 +- docs/userguide/annotations.md | 2 +- docs/userguide/best-practices.md | 2 +- docs/userguide/coverage.md | 2 +- docs/userguide/exception-reporting.md | 2 +- docs/userguide/expectations.md | 2 +- docs/userguide/getting-started.md | 2 +- docs/userguide/install.md | 2 +- docs/userguide/querying_suites.md | 2 +- docs/userguide/reporters.md | 2 +- docs/userguide/running-unit-tests.md | 2 +- docs/userguide/upgrade.md | 2 +- source/core/ut_utils.pks | 2 +- 18 files changed, 18 insertions(+), 18 deletions(-) diff --git a/docs/about/authors.md b/docs/about/authors.md index 9428fddd5..e8d37d574 100644 --- a/docs/about/authors.md +++ b/docs/about/authors.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.13.4068--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.13.4070--develop-blue.svg) ### utPLSQL v3 Major Contributors diff --git a/docs/about/license.md b/docs/about/license.md index d4dbf52cb..5b94283db 100644 --- a/docs/about/license.md +++ b/docs/about/license.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.13.4068--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.13.4070--develop-blue.svg) # Version Information diff --git a/docs/about/project-details.md b/docs/about/project-details.md index 181f9ebe0..26c838e8a 100644 --- a/docs/about/project-details.md +++ b/docs/about/project-details.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.13.4068--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.13.4070--develop-blue.svg) # utPLSQL Project Details diff --git a/docs/about/support.md b/docs/about/support.md index beff5f9cd..d2580f63b 100644 --- a/docs/about/support.md +++ b/docs/about/support.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.13.4068--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.13.4070--develop-blue.svg) # How to get support diff --git a/docs/index.md b/docs/index.md index 4b1219689..0f87193a4 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.13.4068--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.13.4070--develop-blue.svg) ## What is utPLSQL diff --git a/docs/userguide/advanced_data_comparison.md b/docs/userguide/advanced_data_comparison.md index 7f99edbdb..6bb8163e6 100644 --- a/docs/userguide/advanced_data_comparison.md +++ b/docs/userguide/advanced_data_comparison.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.13.4068--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.13.4070--develop-blue.svg) # Advanced data comparison diff --git a/docs/userguide/annotations.md b/docs/userguide/annotations.md index 2272eb321..9913f9a90 100644 --- a/docs/userguide/annotations.md +++ b/docs/userguide/annotations.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.13.4068--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.13.4070--develop-blue.svg) Annotations are used to configure tests and suites in a declarative way similar to modern OOP languages. This way, test configuration is stored along with the test logic inside the test package. No additional configuration files or tables are needed for test cases. The annotation names are based on popular testing frameworks such as JUnit. diff --git a/docs/userguide/best-practices.md b/docs/userguide/best-practices.md index 2eac7a733..94db22808 100644 --- a/docs/userguide/best-practices.md +++ b/docs/userguide/best-practices.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.13.4068--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.13.4070--develop-blue.svg) The following are best practices we at utPLSQL have learned about PL/SQL and Unit Testing. diff --git a/docs/userguide/coverage.md b/docs/userguide/coverage.md index 2e54cfba5..689732e0b 100644 --- a/docs/userguide/coverage.md +++ b/docs/userguide/coverage.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.13.4068--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.13.4070--develop-blue.svg) utPLSQL comes with a built-in coverage reporting engine. The code coverage reporting uses package DBMS_PROFILER (and DBMS_PLSQL_CODE_COVERAGE on Oracle database version 12.2 and above) provided with Oracle database. Code coverage is gathered for the following source types: diff --git a/docs/userguide/exception-reporting.md b/docs/userguide/exception-reporting.md index 43b362e35..81024551e 100644 --- a/docs/userguide/exception-reporting.md +++ b/docs/userguide/exception-reporting.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.13.4068--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.13.4070--develop-blue.svg) utPLSQL is responsible for handling exceptions wherever they occur in the test run. The framework is trapping most of the exceptions so that the test execution is not affected by individual tests or test packages throwing an exception. The framework provides a full stacktrace for every exception that was thrown. The reported stacktrace does not include any utPLSQL library calls in it. diff --git a/docs/userguide/expectations.md b/docs/userguide/expectations.md index bf5d53809..b6854a8cd 100644 --- a/docs/userguide/expectations.md +++ b/docs/userguide/expectations.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.13.4068--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.13.4070--develop-blue.svg) ## Expectation concepts diff --git a/docs/userguide/getting-started.md b/docs/userguide/getting-started.md index 83f80b145..aca1963a7 100644 --- a/docs/userguide/getting-started.md +++ b/docs/userguide/getting-started.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.13.4068--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.13.4070--develop-blue.svg) # Getting started with TDD and utPLSQL diff --git a/docs/userguide/install.md b/docs/userguide/install.md index 9fd89ea83..64d59877d 100644 --- a/docs/userguide/install.md +++ b/docs/userguide/install.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.13.4068--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.13.4070--develop-blue.svg) ## Supported database versions diff --git a/docs/userguide/querying_suites.md b/docs/userguide/querying_suites.md index faef86500..7957d106e 100644 --- a/docs/userguide/querying_suites.md +++ b/docs/userguide/querying_suites.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.13.4068--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.13.4070--develop-blue.svg) ## Obtaining information about suites diff --git a/docs/userguide/reporters.md b/docs/userguide/reporters.md index 3df400c98..ab3a5b56c 100644 --- a/docs/userguide/reporters.md +++ b/docs/userguide/reporters.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.13.4068--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.13.4070--develop-blue.svg) utPLSQL provides several reporting formats. The sections below describe most of them. diff --git a/docs/userguide/running-unit-tests.md b/docs/userguide/running-unit-tests.md index 363ef20b3..dbcfc2b59 100644 --- a/docs/userguide/running-unit-tests.md +++ b/docs/userguide/running-unit-tests.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.13.4068--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.13.4070--develop-blue.svg) utPLSQL framework provides two main entry points to run unit tests from within the database: diff --git a/docs/userguide/upgrade.md b/docs/userguide/upgrade.md index 633346370..f76e86f23 100644 --- a/docs/userguide/upgrade.md +++ b/docs/userguide/upgrade.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.13.4068--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.13.4070--develop-blue.svg) # Upgrading from version 2 diff --git a/source/core/ut_utils.pks b/source/core/ut_utils.pks index 9dba896a9..f965536cb 100644 --- a/source/core/ut_utils.pks +++ b/source/core/ut_utils.pks @@ -21,7 +21,7 @@ create or replace package ut_utils authid definer is * */ - gc_version constant varchar2(50) := 'v3.1.13.4068-develop'; + gc_version constant varchar2(50) := 'v3.1.13.4070-develop'; subtype t_executable_type is varchar2(30); gc_before_all constant t_executable_type := 'beforeall'; From 86cd04dcf436eb1545105f70c4c207d5a78b7bbb Mon Sep 17 00:00:00 2001 From: Jacek Gebal Date: Sun, 11 Dec 2022 22:58:00 +0200 Subject: [PATCH 02/77] Updated releasing.md --- development/releasing.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/development/releasing.md b/development/releasing.md index 4b9e2f7f9..2b9be317a 100644 --- a/development/releasing.md +++ b/development/releasing.md @@ -15,7 +15,10 @@ To create a release follow the below steps - Wait for the [Github Actions `Release`](https://github.com/utPLSQL/utPLSQL/actions/workflows/release.yml) process to complete successfully. The process will upload release artifacts (`zip` and `tar.gz` files along with `md5`) - After Release build was completed successfully, merge the `main` branch back into `develop` branch. At this point, main branch and release tag should be at the same commit version and artifacts should be uploaded into Github release. - After develop branch was built, increase the version number in `VERSION` file to represent next planned release version. - - Clone `utplsql.githug.io` project and add a new announcement about next version being released in `_posts`. Use previous announcements as a template. Make sure to set date, time and post title properly. + - Clone `utplsql.githug.io` project and: + - Add a new announcement about next version being released in `docs/_posts`. Use previous announcements as a template. Make sure to set date, time and post title properly. + - Add the post to list in `mkdocs.yml` file in root directory of that repository. + - Add the link to the post at the beginning of the `docs/index.md` file. The following will happen: - build executed on branch `release/vX.Y.Z-[something]` updates files `sonar-project.properties`, `VERSION` with project version derived from the release branch name From 315e5c6f07e1b3d299bebea722a022ffb60bbab6 Mon Sep 17 00:00:00 2001 From: Jacek Gebal Date: Sat, 21 Jan 2023 00:29:49 +0200 Subject: [PATCH 03/77] Updated version to 3.1.14 --- VERSION | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/VERSION b/VERSION index ecbc73caf..3b63c19c9 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -v3.1.13 +v3.1.14 From f72db1e7dacd09ee273e9df3c65d908db7b3e197 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" Date: Fri, 20 Jan 2023 22:38:37 +0000 Subject: [PATCH 04/77] Updated project version after build [skip ci] --- VERSION | 2 +- docs/about/authors.md | 2 +- docs/about/license.md | 2 +- docs/about/project-details.md | 2 +- docs/about/support.md | 2 +- docs/index.md | 2 +- docs/userguide/advanced_data_comparison.md | 2 +- docs/userguide/annotations.md | 2 +- docs/userguide/best-practices.md | 2 +- docs/userguide/coverage.md | 2 +- docs/userguide/exception-reporting.md | 2 +- docs/userguide/expectations.md | 2 +- docs/userguide/getting-started.md | 2 +- docs/userguide/install.md | 2 +- docs/userguide/querying_suites.md | 2 +- docs/userguide/reporters.md | 2 +- docs/userguide/running-unit-tests.md | 2 +- docs/userguide/upgrade.md | 2 +- sonar-project.properties | 2 +- source/core/ut_utils.pks | 2 +- 20 files changed, 20 insertions(+), 20 deletions(-) diff --git a/VERSION b/VERSION index 3b63c19c9..5fcdcb942 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -v3.1.14 +v3.1.14-develop diff --git a/docs/about/authors.md b/docs/about/authors.md index e8d37d574..cd50c02bf 100644 --- a/docs/about/authors.md +++ b/docs/about/authors.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.13.4070--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4072--develop-blue.svg) ### utPLSQL v3 Major Contributors diff --git a/docs/about/license.md b/docs/about/license.md index 5b94283db..52b771abc 100644 --- a/docs/about/license.md +++ b/docs/about/license.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.13.4070--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4072--develop-blue.svg) # Version Information diff --git a/docs/about/project-details.md b/docs/about/project-details.md index 26c838e8a..c869cfa4f 100644 --- a/docs/about/project-details.md +++ b/docs/about/project-details.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.13.4070--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4072--develop-blue.svg) # utPLSQL Project Details diff --git a/docs/about/support.md b/docs/about/support.md index d2580f63b..ec7316027 100644 --- a/docs/about/support.md +++ b/docs/about/support.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.13.4070--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4072--develop-blue.svg) # How to get support diff --git a/docs/index.md b/docs/index.md index 0f87193a4..4a9108f8a 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.13.4070--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4072--develop-blue.svg) ## What is utPLSQL diff --git a/docs/userguide/advanced_data_comparison.md b/docs/userguide/advanced_data_comparison.md index 6bb8163e6..dc7d2f0ad 100644 --- a/docs/userguide/advanced_data_comparison.md +++ b/docs/userguide/advanced_data_comparison.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.13.4070--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4072--develop-blue.svg) # Advanced data comparison diff --git a/docs/userguide/annotations.md b/docs/userguide/annotations.md index 9913f9a90..87ede5b5f 100644 --- a/docs/userguide/annotations.md +++ b/docs/userguide/annotations.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.13.4070--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4072--develop-blue.svg) Annotations are used to configure tests and suites in a declarative way similar to modern OOP languages. This way, test configuration is stored along with the test logic inside the test package. No additional configuration files or tables are needed for test cases. The annotation names are based on popular testing frameworks such as JUnit. diff --git a/docs/userguide/best-practices.md b/docs/userguide/best-practices.md index 94db22808..905a27acb 100644 --- a/docs/userguide/best-practices.md +++ b/docs/userguide/best-practices.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.13.4070--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4072--develop-blue.svg) The following are best practices we at utPLSQL have learned about PL/SQL and Unit Testing. diff --git a/docs/userguide/coverage.md b/docs/userguide/coverage.md index 689732e0b..5e9331e6b 100644 --- a/docs/userguide/coverage.md +++ b/docs/userguide/coverage.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.13.4070--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4072--develop-blue.svg) utPLSQL comes with a built-in coverage reporting engine. The code coverage reporting uses package DBMS_PROFILER (and DBMS_PLSQL_CODE_COVERAGE on Oracle database version 12.2 and above) provided with Oracle database. Code coverage is gathered for the following source types: diff --git a/docs/userguide/exception-reporting.md b/docs/userguide/exception-reporting.md index 81024551e..0da0f5178 100644 --- a/docs/userguide/exception-reporting.md +++ b/docs/userguide/exception-reporting.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.13.4070--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4072--develop-blue.svg) utPLSQL is responsible for handling exceptions wherever they occur in the test run. The framework is trapping most of the exceptions so that the test execution is not affected by individual tests or test packages throwing an exception. The framework provides a full stacktrace for every exception that was thrown. The reported stacktrace does not include any utPLSQL library calls in it. diff --git a/docs/userguide/expectations.md b/docs/userguide/expectations.md index b6854a8cd..0655700af 100644 --- a/docs/userguide/expectations.md +++ b/docs/userguide/expectations.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.13.4070--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4072--develop-blue.svg) ## Expectation concepts diff --git a/docs/userguide/getting-started.md b/docs/userguide/getting-started.md index aca1963a7..a6948f9d7 100644 --- a/docs/userguide/getting-started.md +++ b/docs/userguide/getting-started.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.13.4070--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4072--develop-blue.svg) # Getting started with TDD and utPLSQL diff --git a/docs/userguide/install.md b/docs/userguide/install.md index 64d59877d..ad3bfe97e 100644 --- a/docs/userguide/install.md +++ b/docs/userguide/install.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.13.4070--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4072--develop-blue.svg) ## Supported database versions diff --git a/docs/userguide/querying_suites.md b/docs/userguide/querying_suites.md index 7957d106e..8cb47b973 100644 --- a/docs/userguide/querying_suites.md +++ b/docs/userguide/querying_suites.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.13.4070--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4072--develop-blue.svg) ## Obtaining information about suites diff --git a/docs/userguide/reporters.md b/docs/userguide/reporters.md index ab3a5b56c..3d0bf7d84 100644 --- a/docs/userguide/reporters.md +++ b/docs/userguide/reporters.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.13.4070--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4072--develop-blue.svg) utPLSQL provides several reporting formats. The sections below describe most of them. diff --git a/docs/userguide/running-unit-tests.md b/docs/userguide/running-unit-tests.md index dbcfc2b59..0ab199c08 100644 --- a/docs/userguide/running-unit-tests.md +++ b/docs/userguide/running-unit-tests.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.13.4070--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4072--develop-blue.svg) utPLSQL framework provides two main entry points to run unit tests from within the database: diff --git a/docs/userguide/upgrade.md b/docs/userguide/upgrade.md index f76e86f23..1a554dfc9 100644 --- a/docs/userguide/upgrade.md +++ b/docs/userguide/upgrade.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.13.4070--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4072--develop-blue.svg) # Upgrading from version 2 diff --git a/sonar-project.properties b/sonar-project.properties index 77cc387a8..d088e9ccb 100644 --- a/sonar-project.properties +++ b/sonar-project.properties @@ -3,7 +3,7 @@ sonar.organization=utplsql sonar.projectKey=utPLSQL # this is the name and version displayed in the SonarQube UI. Was mandatory prior to SonarQube 6.1. sonar.projectName=utPLSQL -sonar.projectVersion=v3.1.13 +sonar.projectVersion=v3.1.14-develop # Path is relative to the sonar-project.properties file. Replace "\" by "/" on Windows. # Since SonarQube 4.2, this property is optional if sonar.modules is set. diff --git a/source/core/ut_utils.pks b/source/core/ut_utils.pks index f965536cb..438abddd2 100644 --- a/source/core/ut_utils.pks +++ b/source/core/ut_utils.pks @@ -21,7 +21,7 @@ create or replace package ut_utils authid definer is * */ - gc_version constant varchar2(50) := 'v3.1.13.4070-develop'; + gc_version constant varchar2(50) := 'v3.1.14.4072-develop'; subtype t_executable_type is varchar2(30); gc_before_all constant t_executable_type := 'beforeall'; From da6ee7089b4dcdab01b7782562f113b3f34ddab0 Mon Sep 17 00:00:00 2001 From: Jacek Gebal Date: Wed, 8 Feb 2023 21:50:46 +0200 Subject: [PATCH 05/77] Refactored output buffers to have better segregation of responsibilities. Changed behavior of output buffer `get_lines` procedure to stop immediately after consuming all produced data if producer has stopped. Changed behavior of output buffer `get_lines` procedure not to timeout after 4 hours, if producer is still running. --- docs/userguide/install.md | 2 +- .../coverage/ut_coverage_reporter_base.tpb | 2 - .../output_buffers/ut_output_buffer_base.tpb | 133 ++++++++++++++++- .../output_buffers/ut_output_buffer_base.tps | 21 ++- .../ut_output_clob_table_buffer.tpb | 134 +++++------------- .../ut_output_clob_table_buffer.tps | 10 +- .../output_buffers/ut_output_table_buffer.tpb | 133 +++++------------ .../output_buffers/ut_output_table_buffer.tps | 10 +- source/core/types/ut_output_reporter_base.tpb | 8 +- source/core/types/ut_output_reporter_base.tps | 3 +- source/create_utplsql_owner.sql | 10 +- test/ut3_tester/core/test_output_buffer.pkb | 25 +++- test/ut3_tester/core/test_output_buffer.pks | 25 +++- test/ut3_tester_helper/coverage_helper.pkb | 19 +-- 14 files changed, 277 insertions(+), 258 deletions(-) diff --git a/docs/userguide/install.md b/docs/userguide/install.md index ad3bfe97e..278bad788 100644 --- a/docs/userguide/install.md +++ b/docs/userguide/install.md @@ -121,7 +121,7 @@ The headless scripts accept three optional parameters that define: The scripts need to be executed by `SYSDBA`, in order to grant access to `DBMS_LOCK` and `DBMS_CRYPTO` system packages. !!! warning "Important" - - Grant on `DBMS_LOCK` is required only for installation on Oracle versions below 18c. For versions 18c and above, utPLSQL uses `DBMS_SESSION.SLEEP` so access to `DBMS_LOCK` package is no longer needed.
+ - `DBMS_LOCK` is required for session synchronization between main session and session consuming realtime reports.
- The user performing the installation must have the `ADMINISTER DATABASE TRIGGER` privilege. This is required for installation of trigger that is responsible for parsing annotations at at compile-time of a package.
- When installed with DDL trigger, utPLSQL will not be registering unit tests for any of oracle-maintained schemas.
- For Oracle 11g following users are excluded:
diff --git a/source/core/coverage/ut_coverage_reporter_base.tpb b/source/core/coverage/ut_coverage_reporter_base.tpb index aff4e6560..da2bd27ad 100644 --- a/source/core/coverage/ut_coverage_reporter_base.tpb +++ b/source/core/coverage/ut_coverage_reporter_base.tpb @@ -92,7 +92,6 @@ create or replace type body ut_coverage_reporter_base is ut_coverage_helper.cleanup_tmp_table(); (l_reporter as ut_output_reporter_base).before_calling_run(null); l_reporter.after_calling_run( ut_run( a_coverage_options => a_coverage_options, a_client_character_set => a_client_character_set ) ); - l_reporter.on_finalize(null); for i in (select /*+ no_parallel */ x.text from table(l_reporter.get_lines(1, 1)) x ) loop pipe row (i.text); end loop; @@ -106,7 +105,6 @@ create or replace type body ut_coverage_reporter_base is ut_coverage_helper.cleanup_tmp_table(); (l_reporter as ut_output_reporter_base).before_calling_run(null); l_reporter.after_calling_run( ut_run( a_coverage_options => a_coverage_options, a_client_character_set => a_client_character_set ) ); - l_reporter.on_finalize(null); open l_result for select /*+ no_parallel */ x.text from table(l_reporter.get_lines(1, 1)) x; return l_result; end; diff --git a/source/core/output_buffers/ut_output_buffer_base.tpb b/source/core/output_buffers/ut_output_buffer_base.tpb index 6d7964150..c3a137b42 100644 --- a/source/core/output_buffers/ut_output_buffer_base.tpb +++ b/source/core/output_buffers/ut_output_buffer_base.tpb @@ -24,7 +24,7 @@ create or replace type body ut_output_buffer_base is self.self_type := coalesce(a_self_type,self.self_type); self.output_id := coalesce(a_output_id, self.output_id, sys_guid()); self.start_date := coalesce(self.start_date, sysdate); - self.last_message_id := 0; + self.last_write_message_id := 0; select /*+ no_parallel */ count(*) into l_exists from ut_output_buffer_info_tmp where output_id = self.output_id; if ( l_exists > 0 ) then update /*+ no_parallel */ ut_output_buffer_info_tmp set start_date = self.start_date where output_id = self.output_id; @@ -32,10 +32,135 @@ create or replace type body ut_output_buffer_base is insert /*+ no_parallel */ into ut_output_buffer_info_tmp(output_id, start_date) values (self.output_id, self.start_date); end if; commit; + dbms_lock.allocate_unique( self.output_id, self.lock_handle); self.is_closed := 0; end; - member function get_lines_cursor(a_initial_timeout natural := null, a_timeout_sec natural := null) return sys_refcursor is + member procedure lock_buffer(a_timeout_sec number := null) is + l_status integer; + begin + l_status := dbms_lock.request( self.lock_handle, dbms_lock.x_mode, 5, false ); + if l_status != 0 then + raise_application_error(-20000, 'Cannot allocate lock for output buffer of reporter. lock request status = '||l_status||', lock handle = '||self.lock_handle||', self.output_id ='||self.output_id); + end if; + end; + + member procedure close(self in out nocopy ut_output_buffer_base) is + l_status integer; + begin + l_status := dbms_lock.release( self.lock_handle ); + if l_status != 0 then + raise_application_error(-20000, 'Cannot release lock for output buffer of reporter. Lock_handle = '||self.lock_handle||' status = '||l_status); + end if; + self.is_closed := 1; + end; + + + member procedure remove_buffer_info(self in ut_output_buffer_base) is + pragma autonomous_transaction; + begin + delete from ut_output_buffer_info_tmp a + where a.output_id = self.output_id; + commit; + end; + + member function get_lines(a_initial_timeout number := null, a_timeout_sec number := null) return ut_output_data_rows pipelined is + lc_init_wait_sec constant number := coalesce(a_initial_timeout, 10 ); + lc_100_milisec constant number(1,1) := 0.1; --sleep for 100 ms between checks + lc_500_milisec constant number(3,1) := 0.5; --sleep for 1 s when waiting long + lc_1_second constant number(3,1) := 1; + l_buffer_rowids ut_varchar2_rows; + l_buffer_data ut_output_data_rows; + l_finished_flags ut_integer_list; + l_last_read_message_id integer; + l_already_waited_sec number(10,2) := 0; + l_data_finished boolean := false; + l_finished boolean := false; + l_sleep_time number(2,1) := lc_100_milisec; + l_lock_status integer; + l_producer_started boolean := false; + l_producer_finished boolean := false; + function get_lock_status return integer is + l_result integer; + l_release_status integer; + begin + l_result := dbms_lock.request( self.lock_handle, dbms_lock.s_mode, 0, false ); + if l_result = 0 then + l_release_status := dbms_lock.release( self.lock_handle ); + end if; + return l_result; + end; + begin + while not l_finished loop + + --check if the lock is still there on output - if yes, the main session is still running and so don't stop + l_lock_status := get_lock_status(); + get_data_from_buffer_table( l_last_read_message_id, l_buffer_data, l_buffer_rowids, l_finished_flags ); + + --nothing fetched from output, wait and try again + if l_buffer_data.count = 0 then + + dbms_lock.sleep(l_sleep_time); + l_already_waited_sec := l_already_waited_sec + l_sleep_time; + + -- if waited more than lc_1_second seconds then increase wait period to minimize the CPU usage. + if l_already_waited_sec >= lc_1_second then + l_sleep_time := lc_500_milisec; + end if; + + else + + l_already_waited_sec := 0; + l_sleep_time := lc_100_milisec; + + for i in 1 .. l_buffer_data.count loop + if l_buffer_data(i).text is not null then + pipe row( l_buffer_data(i) ); + elsif l_finished_flags(i) = 1 then + l_data_finished := true; + exit; + end if; + end loop; + + remove_read_data(l_buffer_rowids); + + end if; + l_producer_started := (l_lock_status <> 0 or l_buffer_data.count > 0) or l_producer_started; + l_producer_finished := (l_producer_started and l_lock_status = 0 and l_buffer_data.count = 0) or l_producer_finished; + + if not l_producer_started and l_already_waited_sec >= lc_init_wait_sec then + + if lc_init_wait_sec > 0 then + self.remove_buffer_info(); + raise_application_error( + ut_utils.gc_out_buffer_timeout, + 'Timeout occurred while waiting for report data producer to start. Waited for: '||ut_utils.to_string( l_already_waited_sec )||' seconds.' + ); + else + l_finished := true; + end if; + + elsif not l_producer_finished and a_timeout_sec is not null and l_already_waited_sec >= a_timeout_sec then + + if a_timeout_sec > 0 then + self.remove_buffer_info(); + raise_application_error( + ut_utils.gc_out_buffer_timeout, + 'Timeout occurred while waiting for more data from producer. Waited for: '||ut_utils.to_string( l_already_waited_sec )||' seconds.' + ); + else + l_finished := true; + end if; + + elsif (l_data_finished or l_producer_finished) then + l_finished := true; + end if; + end loop; + self.remove_buffer_info(); + return; + end; + + member function get_lines_cursor(a_initial_timeout number := null, a_timeout_sec number := null) return sys_refcursor is l_lines sys_refcursor; begin open l_lines for @@ -44,7 +169,7 @@ create or replace type body ut_output_buffer_base is return l_lines; end; - member procedure lines_to_dbms_output(self in ut_output_buffer_base, a_initial_timeout natural := null, a_timeout_sec natural := null) is + member procedure lines_to_dbms_output(self in ut_output_buffer_base, a_initial_timeout number := null, a_timeout_sec number := null) is l_data sys_refcursor; l_clob clob; l_item_type varchar2(32767); @@ -63,7 +188,7 @@ create or replace type body ut_output_buffer_base is end; member procedure cleanup_buffer(self in ut_output_buffer_base, a_retention_time_sec natural := null) is - gc_buffer_retention_sec constant naturaln := coalesce(a_retention_time_sec, 60 * 60 * 24); -- 24 hours + gc_buffer_retention_sec constant naturaln := coalesce(a_retention_time_sec, 60 * 60 * 24 * 5); -- 5 days l_retention_days number := gc_buffer_retention_sec / (60 * 60 * 24); l_max_retention_date date := sysdate - l_retention_days; pragma autonomous_transaction; diff --git a/source/core/output_buffers/ut_output_buffer_base.tps b/source/core/output_buffers/ut_output_buffer_base.tps index 98a6847cd..8000ea2e7 100644 --- a/source/core/output_buffers/ut_output_buffer_base.tps +++ b/source/core/output_buffers/ut_output_buffer_base.tps @@ -19,16 +19,27 @@ create or replace type ut_output_buffer_base force authid definer as object( output_id raw(32), is_closed number(1,0), start_date date, - last_message_id number(38,0), + last_write_message_id number(38,0), + lock_handle varchar2(30 byte), self_type varchar2(250 byte), member procedure init(self in out nocopy ut_output_buffer_base, a_output_id raw := null, a_self_type varchar2 := null), - member function get_lines_cursor(a_initial_timeout natural := null, a_timeout_sec natural := null) return sys_refcursor, - member procedure lines_to_dbms_output(self in ut_output_buffer_base, a_initial_timeout natural := null, a_timeout_sec natural := null), + member procedure lock_buffer(a_timeout_sec number := null), + member function get_lines_cursor(a_initial_timeout number := null, a_timeout_sec number := null) return sys_refcursor, + member procedure lines_to_dbms_output(self in ut_output_buffer_base, a_initial_timeout number := null, a_timeout_sec number := null), member procedure cleanup_buffer(self in ut_output_buffer_base, a_retention_time_sec natural := null), - not instantiable member procedure close(self in out nocopy ut_output_buffer_base), + member procedure remove_buffer_info(self in ut_output_buffer_base), + not instantiable member procedure get_data_from_buffer_table( + self in ut_output_buffer_base, + a_last_read_message_id in out nocopy integer, + a_buffer_data out nocopy ut_output_data_rows, + a_buffer_rowids out nocopy ut_varchar2_rows, + a_finished_flags out nocopy ut_integer_list + ), + member procedure close(self in out nocopy ut_output_buffer_base), + not instantiable member procedure remove_read_data(self in ut_output_buffer_base, a_buffer_rowids ut_varchar2_rows), not instantiable member procedure send_line(self in out nocopy ut_output_buffer_base, a_text varchar2, a_item_type varchar2 := null), not instantiable member procedure send_lines(self in out nocopy ut_output_buffer_base, a_text_list ut_varchar2_rows, a_item_type varchar2 := null), not instantiable member procedure send_clob(self in out nocopy ut_output_buffer_base, a_text clob, a_item_type varchar2 := null), - not instantiable member function get_lines(a_initial_timeout natural := null, a_timeout_sec natural := null) return ut_output_data_rows pipelined + member function get_lines(a_initial_timeout number := null, a_timeout_sec number := null) return ut_output_data_rows pipelined ) not final not instantiable / diff --git a/source/core/output_buffers/ut_output_clob_table_buffer.tpb b/source/core/output_buffers/ut_output_clob_table_buffer.tpb index 66ff71c62..e47f0bcb0 100644 --- a/source/core/output_buffers/ut_output_clob_table_buffer.tpb +++ b/source/core/output_buffers/ut_output_clob_table_buffer.tpb @@ -22,23 +22,13 @@ create or replace type body ut_output_clob_table_buffer is return; end; - overriding member procedure close(self in out nocopy ut_output_clob_table_buffer) is - pragma autonomous_transaction; - begin - self.last_message_id := self.last_message_id + 1; - insert /*+ no_parallel */ into ut_output_clob_buffer_tmp(output_id, message_id, is_finished) - values (self.output_id, self.last_message_id, 1); - commit; - self.is_closed := 1; - end; - overriding member procedure send_line(self in out nocopy ut_output_clob_table_buffer, a_text varchar2, a_item_type varchar2 := null) is pragma autonomous_transaction; begin if a_text is not null or a_item_type is not null then - self.last_message_id := self.last_message_id + 1; + self.last_write_message_id := self.last_write_message_id + 1; insert /*+ no_parallel */ into ut_output_clob_buffer_tmp(output_id, message_id, text, item_type) - values (self.output_id, self.last_message_id, a_text, a_item_type); + values (self.output_id, self.last_write_message_id, a_text, a_item_type); end if; commit; end; @@ -47,10 +37,10 @@ create or replace type body ut_output_clob_table_buffer is pragma autonomous_transaction; begin insert /*+ no_parallel */ into ut_output_clob_buffer_tmp(output_id, message_id, text, item_type) - select /*+ no_parallel */ self.output_id, self.last_message_id + rownum, t.column_value, a_item_type + select /*+ no_parallel */ self.output_id, self.last_write_message_id + rownum, t.column_value, a_item_type from table(a_text_list) t where t.column_value is not null or a_item_type is not null; - self.last_message_id := self.last_message_id + SQL%rowcount; + self.last_write_message_id := self.last_write_message_id + SQL%rowcount; commit; end; @@ -58,99 +48,43 @@ create or replace type body ut_output_clob_table_buffer is pragma autonomous_transaction; begin if a_text is not null and a_text != empty_clob() or a_item_type is not null then - self.last_message_id := self.last_message_id + 1; + self.last_write_message_id := self.last_write_message_id + 1; insert /*+ no_parallel */ into ut_output_clob_buffer_tmp(output_id, message_id, text, item_type) - values (self.output_id, self.last_message_id, a_text, a_item_type); + values (self.output_id, self.last_write_message_id, a_text, a_item_type); end if; commit; end; - overriding member function get_lines(a_initial_timeout natural := null, a_timeout_sec natural := null) return ut_output_data_rows pipelined is - type t_rowid_tab is table of urowid; - l_message_rowids t_rowid_tab; - l_buffer_data ut_output_data_rows; - l_finished_flags ut_integer_list; - l_already_waited_for number(10,2) := 0; - l_finished boolean := false; - lc_init_wait_sec constant naturaln := coalesce(a_initial_timeout, 60 ); -- 1 minute - lc_max_wait_sec constant naturaln := coalesce(a_timeout_sec, 60 * 60 * 4); -- 4 hours - l_wait_for integer := lc_init_wait_sec; - lc_short_sleep_time constant number(1,1) := 0.1; --sleep for 100 ms between checks - lc_long_sleep_time constant number(1) := 1; --sleep for 1 s when waiting long - lc_long_wait_time constant number(1) := 1; --waiting more than 1 sec - l_sleep_time number(2,1) := lc_short_sleep_time; - lc_bulk_limit constant integer := 5000; - l_max_message_id integer := lc_bulk_limit; - - procedure remove_read_data(a_message_rowids t_rowid_tab) is - pragma autonomous_transaction; - begin - forall i in 1 .. a_message_rowids.count - delete from ut_output_clob_buffer_tmp a - where rowid = a_message_rowids(i); - commit; - end; + overriding member procedure get_data_from_buffer_table( + self in ut_output_clob_table_buffer, + a_last_read_message_id in out nocopy integer, + a_buffer_data out nocopy ut_output_data_rows, + a_buffer_rowids out nocopy ut_varchar2_rows, + a_finished_flags out nocopy ut_integer_list + ) is + lc_bulk_limit constant integer := 5000; + begin + a_last_read_message_id := coalesce(a_last_read_message_id, 0); + with ordered_buffer as ( + select /*+ no_parallel index(a) */ ut_output_data_row(a.text, a.item_type), rowidtochar(a.rowid), is_finished + from ut_output_clob_buffer_tmp a + where a.output_id = self.output_id + and a.message_id <= a_last_read_message_id + lc_bulk_limit + order by a.message_id + ) + select /*+ no_parallel */ b.* + bulk collect into a_buffer_data, a_buffer_rowids, a_finished_flags + from ordered_buffer b; + a_last_read_message_id := a_last_read_message_id + a_finished_flags.count; + end; - procedure remove_buffer_info is + overriding member procedure remove_read_data(self in ut_output_clob_table_buffer, a_buffer_rowids ut_varchar2_rows) is pragma autonomous_transaction; - begin - delete from ut_output_buffer_info_tmp a - where a.output_id = self.output_id; - commit; - end; - - begin - while not l_finished loop - with ordered_buffer as ( - select /*+ no_parallel index(a) */ a.rowid, ut_output_data_row(a.text, a.item_type), is_finished - from ut_output_clob_buffer_tmp a - where a.output_id = self.output_id - and a.message_id <= l_max_message_id - order by a.message_id - ) - select /*+ no_parallel */ b.* - bulk collect into l_message_rowids, l_buffer_data, l_finished_flags - from ordered_buffer b; - - --nothing fetched from output, wait and try again - if l_buffer_data.count = 0 then - $if dbms_db_version.version >= 18 $then - dbms_session.sleep(l_sleep_time); - $else - dbms_lock.sleep(l_sleep_time); - $end - l_already_waited_for := l_already_waited_for + l_sleep_time; - if l_already_waited_for > lc_long_wait_time then - l_sleep_time := lc_long_sleep_time; - end if; - else - --reset wait time - -- we wait lc_max_wait_sec for new message - l_wait_for := lc_max_wait_sec; - l_already_waited_for := 0; - l_sleep_time := lc_short_sleep_time; - for i in 1 .. l_buffer_data.count loop - if l_buffer_data(i).text is not null then - pipe row(l_buffer_data(i)); - elsif l_finished_flags(i) = 1 then - l_finished := true; - exit; - end if; - end loop; - remove_read_data(l_message_rowids); - l_max_message_id := l_max_message_id + lc_bulk_limit; - end if; - if l_finished or l_already_waited_for >= l_wait_for then - remove_buffer_info(); - if l_already_waited_for > 0 and l_already_waited_for >= l_wait_for then - raise_application_error( - ut_utils.gc_out_buffer_timeout, - 'Timeout occurred while waiting for output data. Waited for: '||l_already_waited_for||' seconds.' - ); - end if; - end if; - end loop; - return; + begin + forall i in 1 .. a_buffer_rowids.count + delete from ut_output_clob_buffer_tmp a + where rowid = chartorowid(a_buffer_rowids(i)); + commit; end; end; diff --git a/source/core/output_buffers/ut_output_clob_table_buffer.tps b/source/core/output_buffers/ut_output_clob_table_buffer.tps index 7b98efaba..ccd710a8f 100644 --- a/source/core/output_buffers/ut_output_clob_table_buffer.tps +++ b/source/core/output_buffers/ut_output_clob_table_buffer.tps @@ -20,7 +20,13 @@ create or replace type ut_output_clob_table_buffer under ut_output_buffer_base ( overriding member procedure send_line(self in out nocopy ut_output_clob_table_buffer, a_text varchar2, a_item_type varchar2 := null), overriding member procedure send_lines(self in out nocopy ut_output_clob_table_buffer, a_text_list ut_varchar2_rows, a_item_type varchar2 := null), overriding member procedure send_clob(self in out nocopy ut_output_clob_table_buffer, a_text clob, a_item_type varchar2 := null), - overriding member procedure close(self in out nocopy ut_output_clob_table_buffer), - overriding member function get_lines(a_initial_timeout natural := null, a_timeout_sec natural := null) return ut_output_data_rows pipelined + overriding member procedure get_data_from_buffer_table( + self in ut_output_clob_table_buffer, + a_last_read_message_id in out nocopy integer, + a_buffer_data out nocopy ut_output_data_rows, + a_buffer_rowids out nocopy ut_varchar2_rows, + a_finished_flags out nocopy ut_integer_list + ), + overriding member procedure remove_read_data(self in ut_output_clob_table_buffer, a_buffer_rowids ut_varchar2_rows) ) not final / diff --git a/source/core/output_buffers/ut_output_table_buffer.tpb b/source/core/output_buffers/ut_output_table_buffer.tpb index 1809a49d5..189baf075 100644 --- a/source/core/output_buffers/ut_output_table_buffer.tpb +++ b/source/core/output_buffers/ut_output_table_buffer.tpb @@ -22,16 +22,6 @@ create or replace type body ut_output_table_buffer is return; end; - overriding member procedure close(self in out nocopy ut_output_table_buffer) is - pragma autonomous_transaction; - begin - self.last_message_id := self.last_message_id + 1; - insert /*+ no_parallel */ into ut_output_buffer_tmp(output_id, message_id, is_finished) - values (self.output_id, self.last_message_id, 1); - commit; - self.is_closed := 1; - end; - overriding member procedure send_line(self in out nocopy ut_output_table_buffer, a_text varchar2, a_item_type varchar2 := null) is pragma autonomous_transaction; begin @@ -44,9 +34,9 @@ create or replace type body ut_output_table_buffer is a_item_type ); else - self.last_message_id := self.last_message_id + 1; + self.last_write_message_id := self.last_write_message_id + 1; insert /*+ no_parallel */ into ut_output_buffer_tmp(output_id, message_id, text, item_type) - values (self.output_id, self.last_message_id, a_text, a_item_type); + values (self.output_id, self.last_write_message_id, a_text, a_item_type); end if; commit; end if; @@ -56,10 +46,10 @@ create or replace type body ut_output_table_buffer is pragma autonomous_transaction; begin insert /*+ no_parallel */ into ut_output_buffer_tmp(output_id, message_id, text, item_type) - select /*+ no_parallel */ self.output_id, self.last_message_id + rownum, t.column_value, a_item_type + select /*+ no_parallel */ self.output_id, self.last_write_message_id + rownum, t.column_value, a_item_type from table(a_text_list) t where t.column_value is not null or a_item_type is not null; - self.last_message_id := self.last_message_id + SQL%rowcount; + self.last_write_message_id := self.last_write_message_id + SQL%rowcount; commit; end; @@ -75,100 +65,41 @@ create or replace type body ut_output_table_buffer is a_item_type ); else - self.last_message_id := self.last_message_id + 1; + self.last_write_message_id := self.last_write_message_id + 1; insert /*+ no_parallel */ into ut_output_buffer_tmp(output_id, message_id, text, item_type) - values (self.output_id, self.last_message_id, a_text, a_item_type); + values (self.output_id, self.last_write_message_id, a_text, a_item_type); end if; commit; end if; end; - overriding member function get_lines(a_initial_timeout natural := null, a_timeout_sec natural := null) return ut_output_data_rows pipelined is - l_buffer_data ut_varchar2_rows; - l_item_types ut_varchar2_rows; - l_finished_flags ut_integer_list; - l_already_waited_for number(10,2) := 0; - l_finished boolean := false; - lc_init_wait_sec constant naturaln := coalesce(a_initial_timeout, 60 ); -- 1 minute - lc_max_wait_sec constant naturaln := coalesce(a_timeout_sec, 60 * 60 * 4); -- 4 hours - l_wait_for integer := lc_init_wait_sec; - lc_short_sleep_time constant number(1,1) := 0.1; --sleep for 100 ms between checks - lc_long_sleep_time constant number(1) := 1; --sleep for 1 s when waiting long - lc_long_wait_time constant number(1) := 1; --waiting more than 1 sec - l_sleep_time number(2,1) := lc_short_sleep_time; - lc_bulk_limit constant integer := 5000; - l_max_message_id integer := lc_bulk_limit; - - procedure get_data_from_buffer( - a_max_message_id integer, - a_buffer_data out nocopy ut_varchar2_rows, - a_item_types out nocopy ut_varchar2_rows, - a_finished_flags out nocopy ut_integer_list - ) is - pragma autonomous_transaction; - begin - delete /*+ no_parallel */ from ( - select /*+ no_parallel */ * - from ut_output_buffer_tmp o - where o.output_id = self.output_id - and o.message_id <= a_max_message_id - order by o.message_id - ) d - returning d.text, d.item_type, d.is_finished - bulk collect into a_buffer_data, a_item_types, a_finished_flags; - commit; - - end; - - procedure remove_buffer_info is - pragma autonomous_transaction; - begin - delete from ut_output_buffer_info_tmp a - where a.output_id = self.output_id; - commit; - end; + overriding member procedure get_data_from_buffer_table( + self in ut_output_table_buffer, + a_last_read_message_id in out nocopy integer, + a_buffer_data out nocopy ut_output_data_rows, + a_buffer_rowids out nocopy ut_varchar2_rows, + a_finished_flags out nocopy ut_integer_list + ) is + lc_bulk_limit constant integer := 20000; + pragma autonomous_transaction; + begin + a_last_read_message_id := coalesce(a_last_read_message_id,0); + delete /*+ no_parallel */ from ( + select /*+ no_parallel */ * + from ut_output_buffer_tmp o + where o.output_id = self.output_id + and o.message_id <= a_last_read_message_id + lc_bulk_limit + order by o.message_id + ) d + returning ut_output_data_row(d.text, d.item_type), d.is_finished + bulk collect into a_buffer_data, a_finished_flags; + a_last_read_message_id := a_last_read_message_id + a_finished_flags.count; + commit; + end; - begin - while not l_finished loop - get_data_from_buffer( l_max_message_id, l_buffer_data, l_item_types, l_finished_flags); - --nothing fetched from output, wait and try again - if l_buffer_data.count = 0 then - $if dbms_db_version.version >= 18 $then - dbms_session.sleep(l_sleep_time); - $else - dbms_lock.sleep(l_sleep_time); - $end - l_already_waited_for := l_already_waited_for + l_sleep_time; - if l_already_waited_for > lc_long_wait_time then - l_sleep_time := lc_long_sleep_time; - end if; - else - --reset wait time - -- we wait lc_max_wait_sec for new message - l_wait_for := lc_max_wait_sec; - l_already_waited_for := 0; - l_sleep_time := lc_short_sleep_time; - for i in 1 .. l_buffer_data.count loop - if l_buffer_data(i) is not null then - pipe row(ut_output_data_row(l_buffer_data(i),l_item_types(i))); - elsif l_finished_flags(i) = 1 then - l_finished := true; - exit; - end if; - end loop; - l_max_message_id := l_max_message_id + lc_bulk_limit; - end if; - if l_finished or l_already_waited_for >= l_wait_for then - remove_buffer_info(); - if l_already_waited_for > 0 and l_already_waited_for >= l_wait_for then - raise_application_error( - ut_utils.gc_out_buffer_timeout, - 'Timeout occurred while waiting for output data. Waited for: '||l_already_waited_for||' seconds.' - ); - end if; - end if; - end loop; - return; + overriding member procedure remove_read_data(self in ut_output_table_buffer, a_buffer_rowids ut_varchar2_rows) is + begin + null; end; end; diff --git a/source/core/output_buffers/ut_output_table_buffer.tps b/source/core/output_buffers/ut_output_table_buffer.tps index 726b692f8..721e5eb8a 100644 --- a/source/core/output_buffers/ut_output_table_buffer.tps +++ b/source/core/output_buffers/ut_output_table_buffer.tps @@ -20,7 +20,13 @@ create or replace type ut_output_table_buffer under ut_output_buffer_base ( overriding member procedure send_line(self in out nocopy ut_output_table_buffer, a_text varchar2, a_item_type varchar2 := null), overriding member procedure send_lines(self in out nocopy ut_output_table_buffer, a_text_list ut_varchar2_rows, a_item_type varchar2 := null), overriding member procedure send_clob(self in out nocopy ut_output_table_buffer, a_text clob, a_item_type varchar2 := null), - overriding member procedure close(self in out nocopy ut_output_table_buffer), - overriding member function get_lines(a_initial_timeout natural := null, a_timeout_sec natural := null) return ut_output_data_rows pipelined + overriding member procedure get_data_from_buffer_table( + self in ut_output_table_buffer, + a_last_read_message_id in out nocopy integer, + a_buffer_data out nocopy ut_output_data_rows, + a_buffer_rowids out nocopy ut_varchar2_rows, + a_finished_flags out nocopy ut_integer_list + ), + overriding member procedure remove_read_data(self in ut_output_table_buffer, a_buffer_rowids ut_varchar2_rows) ) not final / diff --git a/source/core/types/ut_output_reporter_base.tpb b/source/core/types/ut_output_reporter_base.tpb index f6bb27b94..48970be5a 100644 --- a/source/core/types/ut_output_reporter_base.tpb +++ b/source/core/types/ut_output_reporter_base.tpb @@ -41,13 +41,6 @@ create or replace type body ut_output_reporter_base is return l_result; end; - overriding member procedure before_calling_run(self in out nocopy ut_output_reporter_base, a_run in ut_run) is - l_output_table_buffer ut_output_table_buffer; - begin - (self as ut_reporter_base).before_calling_run(a_run); - l_output_table_buffer := treat(self.output_buffer as ut_output_table_buffer); - end; - member procedure print_text(self in out nocopy ut_output_reporter_base, a_text varchar2, a_item_type varchar2 := null) is begin self.output_buffer.send_line(a_text, a_item_type); @@ -87,6 +80,7 @@ create or replace type body ut_output_reporter_base is overriding member procedure on_initialize(self in out nocopy ut_output_reporter_base, a_run in ut_run) is begin + self.output_buffer.lock_buffer(); self.output_buffer.send_line(null, 'initialize'); end; diff --git a/source/core/types/ut_output_reporter_base.tps b/source/core/types/ut_output_reporter_base.tps index 22f507f8d..21eed9957 100644 --- a/source/core/types/ut_output_reporter_base.tps +++ b/source/core/types/ut_output_reporter_base.tps @@ -20,8 +20,7 @@ create or replace type ut_output_reporter_base under ut_reporter_base( member procedure init(self in out nocopy ut_output_reporter_base, a_self_type varchar2, a_output_buffer ut_output_buffer_base := null), overriding member procedure set_reporter_id(self in out nocopy ut_output_reporter_base, a_reporter_id raw), member function set_reporter_id(self in ut_output_reporter_base, a_reporter_id raw) return ut_output_reporter_base, - overriding member procedure before_calling_run(self in out nocopy ut_output_reporter_base, a_run in ut_run), - + member procedure print_text(self in out nocopy ut_output_reporter_base, a_text varchar2, a_item_type varchar2 := null), member procedure print_text_lines(self in out nocopy ut_output_reporter_base, a_text_lines ut_varchar2_rows, a_item_type varchar2 := null), member procedure print_clob(self in out nocopy ut_output_reporter_base, a_clob clob, a_item_type varchar2 := null), diff --git a/source/create_utplsql_owner.sql b/source/create_utplsql_owner.sql index 64bcb52ce..d7e4f3040 100644 --- a/source/create_utplsql_owner.sql +++ b/source/create_utplsql_owner.sql @@ -31,15 +31,7 @@ create user &ut3_owner_schema identified by "&ut3_password" default tablespace & grant create session, create sequence, create procedure, create type, create table, create view, create synonym to &ut3_owner_schema; -begin - $if dbms_db_version.version < 18 $then - execute immediate 'grant execute on dbms_lock to &ut3_owner_schema'; - $else - null; - $end -end; -/ - +grant execute on dbms_lock to &ut3_owner_schema; grant execute on dbms_crypto to &ut3_owner_schema; grant execute on dbms_lob to &ut3_owner_schema; grant execute on dbms_xmlgen to &ut3_owner_schema; diff --git a/test/ut3_tester/core/test_output_buffer.pkb b/test/ut3_tester/core/test_output_buffer.pkb index 2e8b3337c..c5a4c07b1 100644 --- a/test/ut3_tester/core/test_output_buffer.pkb +++ b/test/ut3_tester/core/test_output_buffer.pkb @@ -16,12 +16,13 @@ create or replace package body test_output_buffer is || chr(13) || chr(10) || to_clob(lpad('a text', 31000, ',a text')) || to_clob(lpad('a text', 31000, ',a text')); l_expected_item_type := lpad('some item type',1000,'-'); --Act + l_buffer.lock_buffer(); l_buffer.send_clob(l_expected_text, l_expected_item_type); l_buffer.close(); select text, item_type into l_actual_text, l_actual_item_type - from table(l_buffer.get_lines(0,0)); + from table(l_buffer.get_lines(0.1,0.1)); --Assert ut.expect(l_actual_text).to_equal(l_expected_text); @@ -32,7 +33,14 @@ create or replace package body test_output_buffer is ut.expect(l_remaining).to_equal(0); end; - + + procedure test_wait_for_producer is + l_buffer ut3_develop.ut_output_buffer_base; + begin + l_buffer := ut3_develop.ut_output_clob_table_buffer(); + ut.expect( l_buffer.get_lines_cursor(0.1,0) ).to_be_empty(); + end; + procedure test_doesnt_send_on_null_text is l_cur sys_refcursor; l_result integer; @@ -86,11 +94,12 @@ create or replace package body test_output_buffer is begin --Arrange l_expected := 'a text'; + l_buffer.lock_buffer(); l_buffer.send_line(l_expected); l_start := localtimestamp; --Act begin - select text into l_result from table(l_buffer.get_lines(1,1)); + select text into l_result from table(l_buffer.get_lines(0,0.3)); ut.fail('Expected a timeout exception but nothing was raised'); exception when others then @@ -101,7 +110,7 @@ create or replace package body test_output_buffer is --Throws a timeout exception ut.expect(dbms_utility.format_error_stack()).to_match('ORA'||ut3_develop.ut_utils.gc_out_buffer_timeout); --Waited for one second - ut.expect(l_duration).to_be_greater_than(interval '0.99' second); + ut.expect(l_duration).to_be_greater_or_equal(interval '0.3' second); end; select count(1) into l_remaining from table(ut3_tester_helper.run_helper.ut_output_buffer_tmp) where output_id = l_buffer.output_id; @@ -116,13 +125,15 @@ create or replace package body test_output_buffer is l_buffer ut3_develop.ut_output_buffer_base; begin --Arrange - l_stale_buffer.start_date := sysdate - 2; + l_stale_buffer.start_date := sysdate - 10; --initialize with new start date l_stale_buffer.init(); + l_stale_buffer.lock_buffer(); l_stale_buffer.send_line('some text'); l_stale_buffer.close(); l_fresh_buffer := ut3_develop.ut_output_table_buffer(); + l_fresh_buffer.lock_buffer(); l_fresh_buffer.send_line('some text'); l_fresh_buffer.close(); @@ -131,9 +142,9 @@ create or replace package body test_output_buffer is --Assert -- Data in "fresh" buffer remains - ut.expect( l_fresh_buffer.get_lines_cursor(0,0), l_buffer.self_type ).to_have_count(1); + ut.expect( l_fresh_buffer.get_lines_cursor(0,0), l_fresh_buffer.self_type ).to_have_count(1); -- Data in "stale" buffer is purged and so the call to get_lines_cursor throws ORA-20218 - ut.expect( l_stale_buffer.get_lines_cursor(0,0), l_buffer.self_type ).to_be_empty(); + ut.expect( l_stale_buffer.get_lines_cursor(0,0), l_stale_buffer.self_type ).to_be_empty(); end; procedure test_purge_text_buffer is diff --git a/test/ut3_tester/core/test_output_buffer.pks b/test/ut3_tester/core/test_output_buffer.pks index 24c2c01eb..feaa337f8 100644 --- a/test/ut3_tester/core/test_output_buffer.pks +++ b/test/ut3_tester/core/test_output_buffer.pks @@ -2,10 +2,33 @@ create or replace package test_output_buffer is --%suite(output_buffer) --%suitepath(utplsql.ut3_tester.core) + + + --%context(Read and write within the same session) + + + --%endcontext + --%context(Buffer is read in a different session than buffer write) + + --reader will wait for a_initial_timeout seconds for the writer process to start and then it will finish with error + + --reader will wait forever (beyond a_initial_timeout) if the writer process is started and end of data row was not received from the buffer + + --reader stops after reading the end of data signal from the buffer + + --reader stops when writer process ends and all data was read from the buffer + + + --%endcontext + --%test(Receives a line from buffer table and deletes) procedure test_receive; + --%test(Waits specified time for producer to lock the buffer ) + --%throws(-20218) + procedure test_wait_for_producer; + --%test(Does not send line if null text given) procedure test_doesnt_send_on_null_text; @@ -19,11 +42,9 @@ create or replace package test_output_buffer is procedure test_waiting_for_data; --%test(Purges text buffer data older than one day and leaves the rest) - --%throws(-20218) procedure test_purge_text_buffer; --%test(Purges clob buffer data older than one day and leaves the rest) - --%throws(-20218) procedure test_purge_clob_buffer; end test_output_buffer; diff --git a/test/ut3_tester_helper/coverage_helper.pkb b/test/ut3_tester_helper/coverage_helper.pkb index 99f26e9f8..3823d9d0c 100644 --- a/test/ut3_tester_helper/coverage_helper.pkb +++ b/test/ut3_tester_helper/coverage_helper.pkb @@ -305,15 +305,6 @@ create or replace package body coverage_helper is return l_status; end; - procedure sleep(a_time number) is - begin - $if dbms_db_version.version >= 18 $then - dbms_session.sleep(a_time); - $else - dbms_lock.sleep(a_time ); - $end - end; - procedure run_job_and_wait_for_finish(a_job_action varchar2) is l_status varchar2(1000); l_job_name varchar2(30); @@ -323,7 +314,7 @@ create or replace package body coverage_helper is begin g_job_no := g_job_no + 1; l_job_name := 'utPLSQL_selftest_job_'||g_job_no; - sleep(0.15); + dbms_lock.sleep(0.15); dbms_scheduler.create_job( job_name => l_job_name, job_type => 'PLSQL_BLOCK', @@ -333,13 +324,13 @@ create or replace package body coverage_helper is auto_drop => TRUE, comments => 'one-time-job' ); - while (l_status is null or l_status not in ('SUCCEEDED','FAILED')) and i < 150 loop + while (l_status is null or l_status not in ('SUCCEEDED','FAILED')) and i < 300 loop l_status := get_job_status( l_job_name, l_timestamp ); - sleep(0.1); + dbms_lock.sleep(0.1); i := i + 1; end loop; commit; - if l_status = 'FAILED' then + if nvl(l_status,'null') <> 'SUCCEEDED' then raise_application_error(-20000, 'Running a scheduler job failed'); end if; end; @@ -378,7 +369,7 @@ create or replace package body coverage_helper is pragma autonomous_transaction; begin run_job_and_wait_for_finish( a_plsql_block ); - + dbms_lock.sleep(0.1); execute immediate q'[ declare l_results ut3_develop.ut_varchar2_list; From a64bd1428499ad207571cd93fcdda78d661d6d92 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jacek=20G=C4=99bal?= Date: Thu, 9 Feb 2023 16:34:53 +0000 Subject: [PATCH 06/77] Update RunExampleTestSuiteWithCompositeReporter.sql --- .../RunExampleTestSuiteWithCompositeReporter.sql | 1 + 1 file changed, 1 insertion(+) diff --git a/examples/developer_examples/RunExampleTestSuiteWithCompositeReporter.sql b/examples/developer_examples/RunExampleTestSuiteWithCompositeReporter.sql index 468d176ed..e713fa02d 100644 --- a/examples/developer_examples/RunExampleTestSuiteWithCompositeReporter.sql +++ b/examples/developer_examples/RunExampleTestSuiteWithCompositeReporter.sql @@ -20,6 +20,7 @@ begin ut_event_manager.initialize(); ut_event_manager.add_listener(l_doc_reporter); ut_event_manager.add_listener(l_tc_reporter); + ut_event_manager.trigger_event(ut_event_manager.gc_initialize, l_run); l_suite := ut_suite(user, 'ut_exampletest',a_line_no=>1); l_suite.description := 'Test Suite Name'; From c1d5ab6c24cb94af1cc22877d31c4b4c13037822 Mon Sep 17 00:00:00 2001 From: Jacek Gebal Date: Fri, 10 Feb 2023 01:31:44 +0200 Subject: [PATCH 07/77] Extended schediler job timeout to 10 minutes to allow for slow execution on older DB versions. We should investigate to see why execution is so slow on 11.2 and 12.1 --- test/ut3_tester_helper/coverage_helper.pkb | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/test/ut3_tester_helper/coverage_helper.pkb b/test/ut3_tester_helper/coverage_helper.pkb index 3823d9d0c..2a508ca6a 100644 --- a/test/ut3_tester_helper/coverage_helper.pkb +++ b/test/ut3_tester_helper/coverage_helper.pkb @@ -290,11 +290,11 @@ create or replace package body coverage_helper is ut3_develop.ut_runner.coverage_stop(); end; - function get_job_status(a_job_name varchar2, a_job_started_after timestamp with time zone) return varchar2 is - l_status varchar2(1000); + function get_job_status(a_job_name varchar2, a_job_started_after timestamp with time zone) return user_scheduler_job_run_details%rowtype is + l_result user_scheduler_job_run_details%rowtype; begin begin - select status into l_status + select * into l_result from user_scheduler_job_run_details where job_name = upper(a_job_name) and req_start_date >= a_job_started_after; @@ -302,11 +302,11 @@ create or replace package body coverage_helper is when no_data_found then null; end; - return l_status; + return l_result; end; procedure run_job_and_wait_for_finish(a_job_action varchar2) is - l_status varchar2(1000); + l_job_run_info user_scheduler_job_run_details%rowtype; l_job_name varchar2(30); l_timestamp timestamp with time zone := current_timestamp; i integer := 0; @@ -324,14 +324,14 @@ create or replace package body coverage_helper is auto_drop => TRUE, comments => 'one-time-job' ); - while (l_status is null or l_status not in ('SUCCEEDED','FAILED')) and i < 300 loop - l_status := get_job_status( l_job_name, l_timestamp ); + while (l_job_run_info.status is null or l_job_run_info.status not in ('SUCCEEDED','FAILED')) and i < 6000 loop + l_job_run_info := get_job_status( l_job_name, l_timestamp ); dbms_lock.sleep(0.1); i := i + 1; end loop; commit; - if nvl(l_status,'null') <> 'SUCCEEDED' then - raise_application_error(-20000, 'Running a scheduler job failed'); + if nvl(l_job_run_info.status,'null') <> 'SUCCEEDED' then + raise_application_error(-20000, 'Scheduler job '''||l_job_name||''', status='''||l_job_run_info.status||'''. Additional info: '||l_job_run_info.additional_info); end if; end; From 99447ed877c9deea0f21cda59ca2f326ad2914be Mon Sep 17 00:00:00 2001 From: Jacek Gebal Date: Fri, 10 Feb 2023 12:35:52 +0200 Subject: [PATCH 08/77] Improving performance of output buffer processing. --- .../core/output_buffers/ut_output_buffer_base.tpb | 12 ++++++++---- .../core/output_buffers/ut_output_table_buffer.tpb | 14 ++++++++++++++ .../core/output_buffers/ut_output_table_buffer.tps | 1 + 3 files changed, 23 insertions(+), 4 deletions(-) diff --git a/source/core/output_buffers/ut_output_buffer_base.tpb b/source/core/output_buffers/ut_output_buffer_base.tpb index c3a137b42..c9cffe339 100644 --- a/source/core/output_buffers/ut_output_buffer_base.tpb +++ b/source/core/output_buffers/ut_output_buffer_base.tpb @@ -179,10 +179,14 @@ create or replace type body ut_output_buffer_base is loop fetch l_data into l_clob, l_item_type; exit when l_data%notfound; - l_lines := ut_utils.clob_to_table(l_clob); - for i in 1 .. l_lines.count loop - dbms_output.put_line(l_lines(i)); - end loop; + if dbms_lob.getlength(l_clob) > 32767 then + l_lines := ut_utils.clob_to_table(l_clob); + for i in 1 .. l_lines.count loop + dbms_output.put_line(l_lines(i)); + end loop; + else + dbms_output.put_line(l_clob); + end if; end loop; close l_data; end; diff --git a/source/core/output_buffers/ut_output_table_buffer.tpb b/source/core/output_buffers/ut_output_table_buffer.tpb index 189baf075..599b56846 100644 --- a/source/core/output_buffers/ut_output_table_buffer.tpb +++ b/source/core/output_buffers/ut_output_table_buffer.tpb @@ -73,6 +73,20 @@ create or replace type body ut_output_table_buffer is end if; end; + overriding member procedure lines_to_dbms_output(self in ut_output_table_buffer, a_initial_timeout number := null, a_timeout_sec number := null) is + l_data sys_refcursor; + l_text varchar2(32767); + l_item_type varchar2(32767); + begin + l_data := self.get_lines_cursor(a_initial_timeout, a_timeout_sec); + loop + fetch l_data into l_text, l_item_type; + exit when l_data%notfound; + dbms_output.put_line(l_text); + end loop; + close l_data; + end; + overriding member procedure get_data_from_buffer_table( self in ut_output_table_buffer, a_last_read_message_id in out nocopy integer, diff --git a/source/core/output_buffers/ut_output_table_buffer.tps b/source/core/output_buffers/ut_output_table_buffer.tps index 721e5eb8a..b7ef8a3e7 100644 --- a/source/core/output_buffers/ut_output_table_buffer.tps +++ b/source/core/output_buffers/ut_output_table_buffer.tps @@ -20,6 +20,7 @@ create or replace type ut_output_table_buffer under ut_output_buffer_base ( overriding member procedure send_line(self in out nocopy ut_output_table_buffer, a_text varchar2, a_item_type varchar2 := null), overriding member procedure send_lines(self in out nocopy ut_output_table_buffer, a_text_list ut_varchar2_rows, a_item_type varchar2 := null), overriding member procedure send_clob(self in out nocopy ut_output_table_buffer, a_text clob, a_item_type varchar2 := null), + overriding member procedure lines_to_dbms_output(self in ut_output_table_buffer, a_initial_timeout number := null, a_timeout_sec number := null), overriding member procedure get_data_from_buffer_table( self in ut_output_table_buffer, a_last_read_message_id in out nocopy integer, From 07f79e29e65e73e38c9219b3188c66f13acb02b4 Mon Sep 17 00:00:00 2001 From: Jacek Gebal Date: Fri, 10 Feb 2023 18:59:22 +0200 Subject: [PATCH 09/77] Fixed performance issues introduced with previous refactoring. --- .../output_buffers/ut_output_buffer_base.tpb | 129 ++++++------------ .../output_buffers/ut_output_buffer_base.tps | 13 +- .../ut_output_clob_table_buffer.tpb | 106 ++++++++++---- .../ut_output_clob_table_buffer.tps | 9 +- .../output_buffers/ut_output_table_buffer.tpb | 69 ++++++++-- .../output_buffers/ut_output_table_buffer.tps | 9 +- 6 files changed, 183 insertions(+), 152 deletions(-) diff --git a/source/core/output_buffers/ut_output_buffer_base.tpb b/source/core/output_buffers/ut_output_buffer_base.tpb index c9cffe339..be4c92bd8 100644 --- a/source/core/output_buffers/ut_output_buffer_base.tpb +++ b/source/core/output_buffers/ut_output_buffer_base.tpb @@ -64,100 +64,51 @@ create or replace type body ut_output_buffer_base is commit; end; - member function get_lines(a_initial_timeout number := null, a_timeout_sec number := null) return ut_output_data_rows pipelined is - lc_init_wait_sec constant number := coalesce(a_initial_timeout, 10 ); - lc_100_milisec constant number(1,1) := 0.1; --sleep for 100 ms between checks - lc_500_milisec constant number(3,1) := 0.5; --sleep for 1 s when waiting long - lc_1_second constant number(3,1) := 1; - l_buffer_rowids ut_varchar2_rows; - l_buffer_data ut_output_data_rows; - l_finished_flags ut_integer_list; - l_last_read_message_id integer; - l_already_waited_sec number(10,2) := 0; - l_data_finished boolean := false; - l_finished boolean := false; - l_sleep_time number(2,1) := lc_100_milisec; - l_lock_status integer; - l_producer_started boolean := false; - l_producer_finished boolean := false; - function get_lock_status return integer is - l_result integer; - l_release_status integer; - begin - l_result := dbms_lock.request( self.lock_handle, dbms_lock.s_mode, 0, false ); - if l_result = 0 then - l_release_status := dbms_lock.release( self.lock_handle ); - end if; - return l_result; - end; + member function timeout_producer_not_started( a_producer_started boolean, a_already_waited_sec number, a_init_wait_sec number ) return boolean + is + l_result boolean := false; begin - while not l_finished loop - - --check if the lock is still there on output - if yes, the main session is still running and so don't stop - l_lock_status := get_lock_status(); - get_data_from_buffer_table( l_last_read_message_id, l_buffer_data, l_buffer_rowids, l_finished_flags ); - - --nothing fetched from output, wait and try again - if l_buffer_data.count = 0 then - - dbms_lock.sleep(l_sleep_time); - l_already_waited_sec := l_already_waited_sec + l_sleep_time; - - -- if waited more than lc_1_second seconds then increase wait period to minimize the CPU usage. - if l_already_waited_sec >= lc_1_second then - l_sleep_time := lc_500_milisec; - end if; - + if not a_producer_started and a_already_waited_sec >= a_init_wait_sec then + if a_init_wait_sec > 0 then + self.remove_buffer_info(); + raise_application_error( + ut_utils.gc_out_buffer_timeout, + 'Timeout occurred while waiting for report data producer to start. Waited for: '||ut_utils.to_string( a_already_waited_sec )||' seconds.' + ); else - - l_already_waited_sec := 0; - l_sleep_time := lc_100_milisec; - - for i in 1 .. l_buffer_data.count loop - if l_buffer_data(i).text is not null then - pipe row( l_buffer_data(i) ); - elsif l_finished_flags(i) = 1 then - l_data_finished := true; - exit; - end if; - end loop; - - remove_read_data(l_buffer_rowids); - + l_result := true; end if; - l_producer_started := (l_lock_status <> 0 or l_buffer_data.count > 0) or l_producer_started; - l_producer_finished := (l_producer_started and l_lock_status = 0 and l_buffer_data.count = 0) or l_producer_finished; - - if not l_producer_started and l_already_waited_sec >= lc_init_wait_sec then - - if lc_init_wait_sec > 0 then - self.remove_buffer_info(); - raise_application_error( - ut_utils.gc_out_buffer_timeout, - 'Timeout occurred while waiting for report data producer to start. Waited for: '||ut_utils.to_string( l_already_waited_sec )||' seconds.' - ); - else - l_finished := true; - end if; - - elsif not l_producer_finished and a_timeout_sec is not null and l_already_waited_sec >= a_timeout_sec then - - if a_timeout_sec > 0 then - self.remove_buffer_info(); - raise_application_error( - ut_utils.gc_out_buffer_timeout, - 'Timeout occurred while waiting for more data from producer. Waited for: '||ut_utils.to_string( l_already_waited_sec )||' seconds.' - ); - else - l_finished := true; - end if; + end if; + return l_result; + end; - elsif (l_data_finished or l_producer_finished) then - l_finished := true; + member function timeout_producer_not_finished(a_producer_finished boolean, a_already_waited_sec number, a_timeout_sec number) return boolean + is + l_result boolean := false; + begin + if not a_producer_finished and a_timeout_sec is not null and a_already_waited_sec >= a_timeout_sec then + if a_timeout_sec > 0 then + self.remove_buffer_info(); + raise_application_error( + ut_utils.gc_out_buffer_timeout, + 'Timeout occurred while waiting for more data from producer. Waited for: '||ut_utils.to_string( a_already_waited_sec )||' seconds.' + ); + else + l_result := true; end if; - end loop; - self.remove_buffer_info(); - return; + end if; + return l_result; + end; + + member function get_lock_status return integer is + l_result integer; + l_release_status integer; + begin + l_result := dbms_lock.request( self.lock_handle, dbms_lock.s_mode, 0, false ); + if l_result = 0 then + l_release_status := dbms_lock.release( self.lock_handle ); + end if; + return l_result; end; member function get_lines_cursor(a_initial_timeout number := null, a_timeout_sec number := null) return sys_refcursor is diff --git a/source/core/output_buffers/ut_output_buffer_base.tps b/source/core/output_buffers/ut_output_buffer_base.tps index 8000ea2e7..53be365ae 100644 --- a/source/core/output_buffers/ut_output_buffer_base.tps +++ b/source/core/output_buffers/ut_output_buffer_base.tps @@ -24,22 +24,17 @@ create or replace type ut_output_buffer_base force authid definer as object( self_type varchar2(250 byte), member procedure init(self in out nocopy ut_output_buffer_base, a_output_id raw := null, a_self_type varchar2 := null), member procedure lock_buffer(a_timeout_sec number := null), + member function timeout_producer_not_started( a_producer_started boolean, a_already_waited_sec number, a_init_wait_sec number ) return boolean, + member function timeout_producer_not_finished(a_producer_finished boolean, a_already_waited_sec number, a_timeout_sec number) return boolean, + member function get_lock_status return integer, member function get_lines_cursor(a_initial_timeout number := null, a_timeout_sec number := null) return sys_refcursor, member procedure lines_to_dbms_output(self in ut_output_buffer_base, a_initial_timeout number := null, a_timeout_sec number := null), member procedure cleanup_buffer(self in ut_output_buffer_base, a_retention_time_sec natural := null), member procedure remove_buffer_info(self in ut_output_buffer_base), - not instantiable member procedure get_data_from_buffer_table( - self in ut_output_buffer_base, - a_last_read_message_id in out nocopy integer, - a_buffer_data out nocopy ut_output_data_rows, - a_buffer_rowids out nocopy ut_varchar2_rows, - a_finished_flags out nocopy ut_integer_list - ), member procedure close(self in out nocopy ut_output_buffer_base), - not instantiable member procedure remove_read_data(self in ut_output_buffer_base, a_buffer_rowids ut_varchar2_rows), not instantiable member procedure send_line(self in out nocopy ut_output_buffer_base, a_text varchar2, a_item_type varchar2 := null), not instantiable member procedure send_lines(self in out nocopy ut_output_buffer_base, a_text_list ut_varchar2_rows, a_item_type varchar2 := null), not instantiable member procedure send_clob(self in out nocopy ut_output_buffer_base, a_text clob, a_item_type varchar2 := null), - member function get_lines(a_initial_timeout number := null, a_timeout_sec number := null) return ut_output_data_rows pipelined + not instantiable member function get_lines(a_initial_timeout number := null, a_timeout_sec number := null) return ut_output_data_rows pipelined ) not final not instantiable / diff --git a/source/core/output_buffers/ut_output_clob_table_buffer.tpb b/source/core/output_buffers/ut_output_clob_table_buffer.tpb index e47f0bcb0..c4cfdb059 100644 --- a/source/core/output_buffers/ut_output_clob_table_buffer.tpb +++ b/source/core/output_buffers/ut_output_clob_table_buffer.tpb @@ -55,36 +55,86 @@ create or replace type body ut_output_clob_table_buffer is commit; end; - overriding member procedure get_data_from_buffer_table( - self in ut_output_clob_table_buffer, - a_last_read_message_id in out nocopy integer, - a_buffer_data out nocopy ut_output_data_rows, - a_buffer_rowids out nocopy ut_varchar2_rows, - a_finished_flags out nocopy ut_integer_list - ) is - lc_bulk_limit constant integer := 5000; - begin - a_last_read_message_id := coalesce(a_last_read_message_id, 0); - with ordered_buffer as ( - select /*+ no_parallel index(a) */ ut_output_data_row(a.text, a.item_type), rowidtochar(a.rowid), is_finished - from ut_output_clob_buffer_tmp a - where a.output_id = self.output_id - and a.message_id <= a_last_read_message_id + lc_bulk_limit - order by a.message_id - ) - select /*+ no_parallel */ b.* - bulk collect into a_buffer_data, a_buffer_rowids, a_finished_flags - from ordered_buffer b; - a_last_read_message_id := a_last_read_message_id + a_finished_flags.count; - end; + overriding member function get_lines(a_initial_timeout number := null, a_timeout_sec number := null) return ut_output_data_rows pipelined is + lc_init_wait_sec constant number := coalesce(a_initial_timeout, 10 ); + l_buffer_rowids ut_varchar2_rows; + l_buffer_data ut_output_data_rows; + l_finished_flags ut_integer_list; + l_last_read_message_id integer; + l_already_waited_sec number(10,2) := 0; + l_finished boolean := false; + l_sleep_time number(2,1); + l_lock_status integer; + l_producer_started boolean := false; + l_producer_finished boolean := false; + procedure get_data_from_buffer_table( + a_last_read_message_id in out nocopy integer, + a_buffer_data out nocopy ut_output_data_rows, + a_buffer_rowids out nocopy ut_varchar2_rows, + a_finished_flags out nocopy ut_integer_list + ) is + lc_bulk_limit constant integer := 5000; + begin + a_last_read_message_id := coalesce(a_last_read_message_id, 0); + with ordered_buffer as ( + select /*+ no_parallel index(a) */ ut_output_data_row(a.text, a.item_type), rowidtochar(a.rowid), is_finished + from ut_output_clob_buffer_tmp a + where a.output_id = self.output_id + and a.message_id <= a_last_read_message_id + lc_bulk_limit + order by a.message_id + ) + select /*+ no_parallel */ b.* + bulk collect into a_buffer_data, a_buffer_rowids, a_finished_flags + from ordered_buffer b; + a_last_read_message_id := a_last_read_message_id + a_finished_flags.count; + end; + + procedure remove_read_data(a_buffer_rowids ut_varchar2_rows) is + pragma autonomous_transaction; + begin + forall i in 1 .. a_buffer_rowids.count + delete from ut_output_clob_buffer_tmp a + where rowid = chartorowid(a_buffer_rowids(i)); + commit; + end; - overriding member procedure remove_read_data(self in ut_output_clob_table_buffer, a_buffer_rowids ut_varchar2_rows) is - pragma autonomous_transaction; begin - forall i in 1 .. a_buffer_rowids.count - delete from ut_output_clob_buffer_tmp a - where rowid = chartorowid(a_buffer_rowids(i)); - commit; + while not l_finished loop + + l_sleep_time := case when l_already_waited_sec >= 1 then 0.5 else 0.1 end; + l_lock_status := self.get_lock_status(); + get_data_from_buffer_table( l_last_read_message_id, l_buffer_data, l_buffer_rowids, l_finished_flags ); + + if l_buffer_data.count > 0 then + l_already_waited_sec := 0; + for i in 1 .. l_buffer_data.count loop + if l_buffer_data(i).text is not null then + pipe row( l_buffer_data(i) ); + elsif l_finished_flags(i) = 1 then + l_finished := true; + exit; + end if; + end loop; + remove_read_data(l_buffer_rowids); + else + --nothing fetched from output, wait. + dbms_lock.sleep(l_sleep_time); + l_already_waited_sec := l_already_waited_sec + l_sleep_time; + end if; + + l_producer_started := (l_lock_status <> 0 or l_buffer_data.count > 0) or l_producer_started; + l_producer_finished := (l_producer_started and l_lock_status = 0 and l_buffer_data.count = 0) or l_producer_finished; + + l_finished := + self.timeout_producer_not_finished(l_producer_finished, l_already_waited_sec, a_timeout_sec) + or self.timeout_producer_not_started(l_producer_started, l_already_waited_sec, lc_init_wait_sec) + or l_producer_finished + or l_finished; + + end loop; + + self.remove_buffer_info(); + return; end; end; diff --git a/source/core/output_buffers/ut_output_clob_table_buffer.tps b/source/core/output_buffers/ut_output_clob_table_buffer.tps index ccd710a8f..191e64c01 100644 --- a/source/core/output_buffers/ut_output_clob_table_buffer.tps +++ b/source/core/output_buffers/ut_output_clob_table_buffer.tps @@ -20,13 +20,6 @@ create or replace type ut_output_clob_table_buffer under ut_output_buffer_base ( overriding member procedure send_line(self in out nocopy ut_output_clob_table_buffer, a_text varchar2, a_item_type varchar2 := null), overriding member procedure send_lines(self in out nocopy ut_output_clob_table_buffer, a_text_list ut_varchar2_rows, a_item_type varchar2 := null), overriding member procedure send_clob(self in out nocopy ut_output_clob_table_buffer, a_text clob, a_item_type varchar2 := null), - overriding member procedure get_data_from_buffer_table( - self in ut_output_clob_table_buffer, - a_last_read_message_id in out nocopy integer, - a_buffer_data out nocopy ut_output_data_rows, - a_buffer_rowids out nocopy ut_varchar2_rows, - a_finished_flags out nocopy ut_integer_list - ), - overriding member procedure remove_read_data(self in ut_output_clob_table_buffer, a_buffer_rowids ut_varchar2_rows) + overriding member function get_lines(a_initial_timeout number := null, a_timeout_sec number := null) return ut_output_data_rows pipelined ) not final / diff --git a/source/core/output_buffers/ut_output_table_buffer.tpb b/source/core/output_buffers/ut_output_table_buffer.tpb index 599b56846..f38363e49 100644 --- a/source/core/output_buffers/ut_output_table_buffer.tpb +++ b/source/core/output_buffers/ut_output_table_buffer.tpb @@ -87,12 +87,29 @@ create or replace type body ut_output_table_buffer is close l_data; end; - overriding member procedure get_data_from_buffer_table( - self in ut_output_table_buffer, + /* Important note. + This function code is almost duplicated between two types for performance reasons. + The pipe row clause is much faster on VARCHAR2 then it is on clob. + That is the key reason for two implementations. + */ + overriding member function get_lines(a_initial_timeout number := null, a_timeout_sec number := null) return ut_output_data_rows pipelined is + lc_init_wait_sec constant number := coalesce(a_initial_timeout, 10 ); + l_buffer_texts ut_varchar2_rows; + l_buffer_item_types ut_varchar2_rows; + l_finished_flags ut_integer_list; + l_last_read_message_id integer; + l_already_waited_sec number(10,2) := 0; + l_finished boolean := false; + l_sleep_time number(2,1); + l_lock_status integer; + l_producer_started boolean := false; + l_producer_finished boolean := false; + + procedure get_data_from_buffer_table( a_last_read_message_id in out nocopy integer, - a_buffer_data out nocopy ut_output_data_rows, - a_buffer_rowids out nocopy ut_varchar2_rows, - a_finished_flags out nocopy ut_integer_list + a_buffer_texts out nocopy ut_varchar2_rows, + a_buffer_item_types out nocopy ut_varchar2_rows, + a_finished_flags out nocopy ut_integer_list ) is lc_bulk_limit constant integer := 20000; pragma autonomous_transaction; @@ -105,15 +122,47 @@ create or replace type body ut_output_table_buffer is and o.message_id <= a_last_read_message_id + lc_bulk_limit order by o.message_id ) d - returning ut_output_data_row(d.text, d.item_type), d.is_finished - bulk collect into a_buffer_data, a_finished_flags; + returning d.text, d.item_type, d.is_finished + bulk collect into a_buffer_texts, a_buffer_item_types, a_finished_flags; a_last_read_message_id := a_last_read_message_id + a_finished_flags.count; commit; end; - - overriding member procedure remove_read_data(self in ut_output_table_buffer, a_buffer_rowids ut_varchar2_rows) is begin - null; + while not l_finished loop + + l_sleep_time := case when l_already_waited_sec >= 1 then 0.5 else 0.1 end; + l_lock_status := self.get_lock_status(); + get_data_from_buffer_table( l_last_read_message_id, l_buffer_texts, l_buffer_item_types, l_finished_flags ); + + if l_buffer_texts.count > 0 then + l_already_waited_sec := 0; + for i in 1 .. l_buffer_texts.count loop + if l_buffer_texts(i) is not null then + pipe row( ut_output_data_row(l_buffer_texts(i), l_buffer_item_types(i)) ); + elsif l_finished_flags(i) = 1 then + l_finished := true; + exit; + end if; + end loop; + else + --nothing fetched from output, wait. + dbms_lock.sleep(l_sleep_time); + l_already_waited_sec := l_already_waited_sec + l_sleep_time; + end if; + + l_producer_started := (l_lock_status <> 0 or l_buffer_texts.count > 0) or l_producer_started; + l_producer_finished := (l_producer_started and l_lock_status = 0 and l_buffer_texts.count = 0) or l_producer_finished; + + l_finished := + self.timeout_producer_not_finished(l_producer_finished, l_already_waited_sec, a_timeout_sec) + or self.timeout_producer_not_started(l_producer_started, l_already_waited_sec, lc_init_wait_sec) + or l_producer_finished + or l_finished; + + end loop; + + self.remove_buffer_info(); + return; end; end; diff --git a/source/core/output_buffers/ut_output_table_buffer.tps b/source/core/output_buffers/ut_output_table_buffer.tps index b7ef8a3e7..154ce4de6 100644 --- a/source/core/output_buffers/ut_output_table_buffer.tps +++ b/source/core/output_buffers/ut_output_table_buffer.tps @@ -21,13 +21,6 @@ create or replace type ut_output_table_buffer under ut_output_buffer_base ( overriding member procedure send_lines(self in out nocopy ut_output_table_buffer, a_text_list ut_varchar2_rows, a_item_type varchar2 := null), overriding member procedure send_clob(self in out nocopy ut_output_table_buffer, a_text clob, a_item_type varchar2 := null), overriding member procedure lines_to_dbms_output(self in ut_output_table_buffer, a_initial_timeout number := null, a_timeout_sec number := null), - overriding member procedure get_data_from_buffer_table( - self in ut_output_table_buffer, - a_last_read_message_id in out nocopy integer, - a_buffer_data out nocopy ut_output_data_rows, - a_buffer_rowids out nocopy ut_varchar2_rows, - a_finished_flags out nocopy ut_integer_list - ), - overriding member procedure remove_read_data(self in ut_output_table_buffer, a_buffer_rowids ut_varchar2_rows) + overriding member function get_lines(a_initial_timeout number := null, a_timeout_sec number := null) return ut_output_data_rows pipelined ) not final / From 865785aa49c0258f9b1c1ad8291d07867300daf6 Mon Sep 17 00:00:00 2001 From: Jacek Gebal Date: Fri, 10 Feb 2023 19:18:31 +0200 Subject: [PATCH 10/77] Fixed failing test. --- development/refresh_sources.sh | 2 +- test/ut3_tester/core/test_output_buffer.pkb | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/development/refresh_sources.sh b/development/refresh_sources.sh index b6ef51ef9..83518d7cc 100755 --- a/development/refresh_sources.sh +++ b/development/refresh_sources.sh @@ -12,7 +12,7 @@ git clone --depth=1 --branch=${SELFTESTING_BRANCH:-main} https://github.com/utPL rm -rf utPLSQL-cli/* # download latest release version of utPLSQL-cli -curl -Lk -o utPLSQL-cli.zip https://github.com/utPLSQL/utPLSQL-cli/releases/download/v${UTPLSQL_CLI_VERSION}/utPLSQL-cli.zip +curl -Lk -o utPLSQL-cli.zip https://github.com/utPLSQL/utPLSQL-cli/releases/download/${UTPLSQL_CLI_VERSION}/utPLSQL-cli.zip # unzip utPLSQL-cli and remove the zip file unzip utPLSQL-cli.zip && chmod u+x utPLSQL-cli/bin/utplsql && rm utPLSQL-cli.zip diff --git a/test/ut3_tester/core/test_output_buffer.pkb b/test/ut3_tester/core/test_output_buffer.pkb index c5a4c07b1..edb10e3e6 100644 --- a/test/ut3_tester/core/test_output_buffer.pkb +++ b/test/ut3_tester/core/test_output_buffer.pkb @@ -38,7 +38,7 @@ create or replace package body test_output_buffer is l_buffer ut3_develop.ut_output_buffer_base; begin l_buffer := ut3_develop.ut_output_clob_table_buffer(); - ut.expect( l_buffer.get_lines_cursor(0.1,0) ).to_be_empty(); + ut.expect( l_buffer.get_lines_cursor(0.1) ).to_be_empty(); end; procedure test_doesnt_send_on_null_text is From 33f5485645d074f6993402a4f11c8f350534dd43 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" Date: Sat, 11 Feb 2023 10:57:45 +0000 Subject: [PATCH 11/77] Updated project version after build [skip ci] --- docs/about/authors.md | 2 +- docs/about/license.md | 2 +- docs/about/project-details.md | 2 +- docs/about/support.md | 2 +- docs/index.md | 2 +- docs/userguide/advanced_data_comparison.md | 2 +- docs/userguide/annotations.md | 2 +- docs/userguide/best-practices.md | 2 +- docs/userguide/coverage.md | 2 +- docs/userguide/exception-reporting.md | 2 +- docs/userguide/expectations.md | 2 +- docs/userguide/getting-started.md | 2 +- docs/userguide/install.md | 2 +- docs/userguide/querying_suites.md | 2 +- docs/userguide/reporters.md | 2 +- docs/userguide/running-unit-tests.md | 2 +- docs/userguide/upgrade.md | 2 +- source/core/ut_utils.pks | 2 +- 18 files changed, 18 insertions(+), 18 deletions(-) diff --git a/docs/about/authors.md b/docs/about/authors.md index cd50c02bf..9bd1a10c6 100644 --- a/docs/about/authors.md +++ b/docs/about/authors.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4072--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4085--develop-blue.svg) ### utPLSQL v3 Major Contributors diff --git a/docs/about/license.md b/docs/about/license.md index 52b771abc..727b513eb 100644 --- a/docs/about/license.md +++ b/docs/about/license.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4072--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4085--develop-blue.svg) # Version Information diff --git a/docs/about/project-details.md b/docs/about/project-details.md index c869cfa4f..5f4b1d57c 100644 --- a/docs/about/project-details.md +++ b/docs/about/project-details.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4072--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4085--develop-blue.svg) # utPLSQL Project Details diff --git a/docs/about/support.md b/docs/about/support.md index ec7316027..cd2f76d49 100644 --- a/docs/about/support.md +++ b/docs/about/support.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4072--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4085--develop-blue.svg) # How to get support diff --git a/docs/index.md b/docs/index.md index 4a9108f8a..b029f6206 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4072--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4085--develop-blue.svg) ## What is utPLSQL diff --git a/docs/userguide/advanced_data_comparison.md b/docs/userguide/advanced_data_comparison.md index dc7d2f0ad..7a2f511aa 100644 --- a/docs/userguide/advanced_data_comparison.md +++ b/docs/userguide/advanced_data_comparison.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4072--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4085--develop-blue.svg) # Advanced data comparison diff --git a/docs/userguide/annotations.md b/docs/userguide/annotations.md index 87ede5b5f..0e3b6446e 100644 --- a/docs/userguide/annotations.md +++ b/docs/userguide/annotations.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4072--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4085--develop-blue.svg) Annotations are used to configure tests and suites in a declarative way similar to modern OOP languages. This way, test configuration is stored along with the test logic inside the test package. No additional configuration files or tables are needed for test cases. The annotation names are based on popular testing frameworks such as JUnit. diff --git a/docs/userguide/best-practices.md b/docs/userguide/best-practices.md index 905a27acb..d41641303 100644 --- a/docs/userguide/best-practices.md +++ b/docs/userguide/best-practices.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4072--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4085--develop-blue.svg) The following are best practices we at utPLSQL have learned about PL/SQL and Unit Testing. diff --git a/docs/userguide/coverage.md b/docs/userguide/coverage.md index 5e9331e6b..5e1c4cbdb 100644 --- a/docs/userguide/coverage.md +++ b/docs/userguide/coverage.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4072--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4085--develop-blue.svg) utPLSQL comes with a built-in coverage reporting engine. The code coverage reporting uses package DBMS_PROFILER (and DBMS_PLSQL_CODE_COVERAGE on Oracle database version 12.2 and above) provided with Oracle database. Code coverage is gathered for the following source types: diff --git a/docs/userguide/exception-reporting.md b/docs/userguide/exception-reporting.md index 0da0f5178..8c986cb87 100644 --- a/docs/userguide/exception-reporting.md +++ b/docs/userguide/exception-reporting.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4072--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4085--develop-blue.svg) utPLSQL is responsible for handling exceptions wherever they occur in the test run. The framework is trapping most of the exceptions so that the test execution is not affected by individual tests or test packages throwing an exception. The framework provides a full stacktrace for every exception that was thrown. The reported stacktrace does not include any utPLSQL library calls in it. diff --git a/docs/userguide/expectations.md b/docs/userguide/expectations.md index 0655700af..c116464de 100644 --- a/docs/userguide/expectations.md +++ b/docs/userguide/expectations.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4072--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4085--develop-blue.svg) ## Expectation concepts diff --git a/docs/userguide/getting-started.md b/docs/userguide/getting-started.md index a6948f9d7..4427f2f10 100644 --- a/docs/userguide/getting-started.md +++ b/docs/userguide/getting-started.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4072--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4085--develop-blue.svg) # Getting started with TDD and utPLSQL diff --git a/docs/userguide/install.md b/docs/userguide/install.md index 278bad788..7af30194d 100644 --- a/docs/userguide/install.md +++ b/docs/userguide/install.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4072--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4085--develop-blue.svg) ## Supported database versions diff --git a/docs/userguide/querying_suites.md b/docs/userguide/querying_suites.md index 8cb47b973..68c689828 100644 --- a/docs/userguide/querying_suites.md +++ b/docs/userguide/querying_suites.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4072--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4085--develop-blue.svg) ## Obtaining information about suites diff --git a/docs/userguide/reporters.md b/docs/userguide/reporters.md index 3d0bf7d84..75bd4df3d 100644 --- a/docs/userguide/reporters.md +++ b/docs/userguide/reporters.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4072--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4085--develop-blue.svg) utPLSQL provides several reporting formats. The sections below describe most of them. diff --git a/docs/userguide/running-unit-tests.md b/docs/userguide/running-unit-tests.md index 0ab199c08..07b192ce1 100644 --- a/docs/userguide/running-unit-tests.md +++ b/docs/userguide/running-unit-tests.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4072--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4085--develop-blue.svg) utPLSQL framework provides two main entry points to run unit tests from within the database: diff --git a/docs/userguide/upgrade.md b/docs/userguide/upgrade.md index 1a554dfc9..bb66e6e0e 100644 --- a/docs/userguide/upgrade.md +++ b/docs/userguide/upgrade.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4072--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4085--develop-blue.svg) # Upgrading from version 2 diff --git a/source/core/ut_utils.pks b/source/core/ut_utils.pks index 438abddd2..18a3d5952 100644 --- a/source/core/ut_utils.pks +++ b/source/core/ut_utils.pks @@ -21,7 +21,7 @@ create or replace package ut_utils authid definer is * */ - gc_version constant varchar2(50) := 'v3.1.14.4072-develop'; + gc_version constant varchar2(50) := 'v3.1.14.4085-develop'; subtype t_executable_type is varchar2(30); gc_before_all constant t_executable_type := 'beforeall'; From 28b6eaa53ec15e0fc11c8ff2531388d7e857c645 Mon Sep 17 00:00:00 2001 From: Jacek Gebal Date: Sat, 11 Feb 2023 12:54:42 +0200 Subject: [PATCH 12/77] Small performance improvements to output buffer handling and some tests. --- source/api/ut.pkb | 12 +++++++++--- .../core/output_buffers/ut_output_table_buffer.tpb | 10 ++++++---- test/ut3_tester_helper/coverage_helper.pkb | 14 ++++++++++++++ 3 files changed, 29 insertions(+), 7 deletions(-) diff --git a/source/api/ut.pkb b/source/api/ut.pkb index 5c91e54d4..5e612efb4 100644 --- a/source/api/ut.pkb +++ b/source/api/ut.pkb @@ -222,10 +222,15 @@ create or replace package body ut is raise_if_packages_invalidated(); raise no_data_found; end if; - g_result_lines := ut_utils.clob_to_table(l_clob, ut_utils.gc_max_storage_varchar2_len); - g_result_line_no := g_result_lines.first; + if l_clob is not null and l_clob != empty_clob() then + if length(l_clob) > ut_utils.gc_max_storage_varchar2_len then + g_result_lines := ut_utils.clob_to_table(l_clob, ut_utils.gc_max_storage_varchar2_len); + else + g_result_lines := ut_varchar2_list(l_clob); + end if; + g_result_line_no := g_result_lines.first; + end if; end if; - if g_result_line_no is not null then l_result := g_result_lines(g_result_line_no); g_result_line_no := g_result_lines.next(g_result_line_no); @@ -274,6 +279,7 @@ create or replace package body ut is if l_reporter is of (ut_output_reporter_base) then l_results := treat(l_reporter as ut_output_reporter_base).get_lines_cursor(); loop + g_result_lines := ut_varchar2_list(); pipe row( get_report_outputs( l_results ) ); end loop; end if; diff --git a/source/core/output_buffers/ut_output_table_buffer.tpb b/source/core/output_buffers/ut_output_table_buffer.tpb index f38363e49..480ae9144 100644 --- a/source/core/output_buffers/ut_output_table_buffer.tpb +++ b/source/core/output_buffers/ut_output_table_buffer.tpb @@ -75,14 +75,16 @@ create or replace type body ut_output_table_buffer is overriding member procedure lines_to_dbms_output(self in ut_output_table_buffer, a_initial_timeout number := null, a_timeout_sec number := null) is l_data sys_refcursor; - l_text varchar2(32767); - l_item_type varchar2(32767); + l_text ut_varchar2_rows; + l_item_type ut_varchar2_rows; begin l_data := self.get_lines_cursor(a_initial_timeout, a_timeout_sec); loop - fetch l_data into l_text, l_item_type; + fetch l_data bulk collect into l_text, l_item_type limit 10000; + for idx in 1 .. l_text.count loop + dbms_output.put_line(l_text(idx)); + end loop; exit when l_data%notfound; - dbms_output.put_line(l_text); end loop; close l_data; end; diff --git a/test/ut3_tester_helper/coverage_helper.pkb b/test/ut3_tester_helper/coverage_helper.pkb index 2a508ca6a..d234967e4 100644 --- a/test/ut3_tester_helper/coverage_helper.pkb +++ b/test/ut3_tester_helper/coverage_helper.pkb @@ -489,16 +489,30 @@ create or replace package body coverage_helper is l_coverage_id raw(32) := sys_guid(); begin l_plsql_block := q'[ + declare + x dbms_output.chararr; + i integer := 100000; begin + execute immediate 'alter session set statistics_level=all'; +/* + dbms_hprof.start_profiling( + location => 'PLSHPROF_DIR' + , filename => 'profiler_utPLSQL_run_]'||rawtohex(l_coverage_id)||q'[.txt' + ); +*/ ut3_develop.ut_runner.coverage_start(']'||rawtohex(l_coverage_id)||q'['); ut3_develop.ut_coverage.set_develop_mode(a_develop_mode => true); --gather coverage on the command executed begin {a_run_command}; end; + dbms_output.get_lines(x,i); ut3_develop.ut_coverage.set_develop_mode(a_develop_mode => false); ut3_develop.ut_runner.coverage_stop(); --get the actual results of the command gathering the coverage insert into test_results select rownum as id, x.* from table( {a_run_command} ) x; commit; +/* + dbms_hprof.stop_profiling; +*/ end;]'; l_plsql_block := replace(l_plsql_block,'{a_run_command}',a_run_command); l_result_clob := run_code_as_job( l_plsql_block ); From 3a837f4ccf5021fbbb8d2a269653226601ca535d Mon Sep 17 00:00:00 2001 From: Lukasz Wasylow Date: Tue, 14 Mar 2023 23:10:15 -0700 Subject: [PATCH 13/77] Address issue where the not_to(contain) executes a negated matcher without negated flag set causing refcursor compare sql generated inclusion comparision. --- source/expectations/matchers/ut_contain.tpb | 1 + source/expectations/ut_expectation.tpb | 4 +- .../expectations/test_expectations_cursor.pkb | 100 +++++++++++++++++- .../expectations/test_expectations_cursor.pks | 22 +++- 4 files changed, 122 insertions(+), 5 deletions(-) diff --git a/source/expectations/matchers/ut_contain.tpb b/source/expectations/matchers/ut_contain.tpb index 7591925f5..c9691f731 100644 --- a/source/expectations/matchers/ut_contain.tpb +++ b/source/expectations/matchers/ut_contain.tpb @@ -47,6 +47,7 @@ create or replace type body ut_contain as overriding member function run_matcher_negated(self in out nocopy ut_contain, a_actual ut_data_value) return boolean is begin + self.negated(); return run_matcher(a_actual); end; diff --git a/source/expectations/ut_expectation.tpb b/source/expectations/ut_expectation.tpb index a78c57eb7..309759d48 100644 --- a/source/expectations/ut_expectation.tpb +++ b/source/expectations/ut_expectation.tpb @@ -686,7 +686,7 @@ create or replace type body ut_expectation as member procedure not_to_contain(self in ut_expectation, a_expected sys_refcursor) is begin - self.not_to( ut_contain(a_expected).negated() ); + self.not_to( ut_contain(a_expected)); end; member procedure to_contain(self in ut_expectation, a_expected anydata) is @@ -696,7 +696,7 @@ create or replace type body ut_expectation as member procedure not_to_contain(self in ut_expectation, a_expected anydata) is begin - self.not_to( ut_contain(a_expected).negated() ); + self.not_to( ut_contain(a_expected)); end; member function to_be_within(a_dist number) return ut_be_within is diff --git a/test/ut3_user/expectations/test_expectations_cursor.pkb b/test/ut3_user/expectations/test_expectations_cursor.pkb index 952084fd3..7d0f308a8 100644 --- a/test/ut3_user/expectations/test_expectations_cursor.pkb +++ b/test/ut3_user/expectations/test_expectations_cursor.pkb @@ -2329,6 +2329,23 @@ Diff:% ut.expect(ut3_tester_helper.main_helper.get_failed_expectations_num).to_equal(0); end; + procedure cursor_not_to_contain2 + as + l_actual sys_refcursor; + l_expected sys_refcursor; + begin + open l_expected for select 'TEST' username, -600 user_id from dual; + + open l_actual for select username, user_id from all_users + union all + select 'TEST1' username, -601 user_id from dual; + + --Act + ut3_develop.ut.expect(l_actual).not_to(ut3_develop.contain(l_expected)); + --Assert + ut.expect(ut3_tester_helper.main_helper.get_failed_expectations_num).to_equal(0); + end; + procedure cursor_not_to_contain_fail is l_actual sys_refcursor; l_expected sys_refcursor; @@ -2359,6 +2376,37 @@ Diff:% ut.expect(l_actual_message).to_be_like(l_expected_message); end; + + procedure cursor_not_to_contain_fail2 is + l_actual sys_refcursor; + l_expected sys_refcursor; + l_expected_message varchar2(32767); + l_actual_message varchar2(32767); + begin + --Arrange + open l_expected for select 'TEST' username, -600 user_id from dual; + + open l_actual for select username, user_id from all_users + union all + select 'TEST' username, -600 user_id from dual; + + --Act + ut3_develop.ut.expect(l_actual).not_to(ut3_develop.contain(l_expected)); + --Assert + l_expected_message := q'[%Actual: (refcursor [ count = % ])% +%Data-types:% +%VARCHAR2NUMBER% +%Data:% +%was expected not to contain:(refcursor [ count = 1 ])% +%Data-types:% +%CHARNUMBER% +%Data:% +%TEST-600%]'; + l_actual_message := ut3_tester_helper.main_helper.get_failed_expectations(1); + --Assert + ut.expect(l_actual_message).to_be_like(l_expected_message); + end; + procedure cursor_not_to_contain_joinby is l_actual sys_refcursor; l_expected sys_refcursor; @@ -2372,7 +2420,21 @@ Diff:% --Assert ut.expect(ut3_tester_helper.main_helper.get_failed_expectations_num).to_equal(0); end; - + + procedure cursor_not_to_contain_joinby2 is + l_actual sys_refcursor; + l_expected sys_refcursor; + begin + --Arrange + open l_actual for select username,rownum * 10 user_id from all_users where rownum < 5; + open l_expected for select username||to_char(rownum) username ,rownum user_id from all_users where rownum < 5; + + --Act + ut3_develop.ut.expect(l_actual).not_to(ut3_develop.contain(l_expected).join_by('USER_ID')); + --Assert + ut.expect(ut3_tester_helper.main_helper.get_failed_expectations_num).to_equal(0); + end; + procedure not_cont_join_incl_cols_as_lst is l_actual sys_refcursor; l_expected sys_refcursor; @@ -2386,6 +2448,19 @@ Diff:% ut.expect(ut3_tester_helper.main_helper.get_failed_expectations_num).to_equal(0); end; + procedure not_cont_join_incl_cols_as_lst2 is + l_actual sys_refcursor; + l_expected sys_refcursor; + begin + --Arrange + open l_actual for select rownum as rn, 'b' as "A_Column", 'c' as A_COLUMN, 'x' SOME_COL, 'd' "Some_Col" from dual a connect by level < 10; + open l_expected for select rownum * 20 rn, 'a' as "A_Column", 'd' as A_COLUMN, 'x' SOME_COL, 'c' "Some_Col" from dual a connect by level < 4; + --Act + ut3_develop.ut.expect(l_actual).not_to(ut3_develop.contain(l_expected).include(ut3_develop.ut_varchar2_list('RN','//A_Column','SOME_COL')).join_by('RN')); + --Assert + ut.expect(ut3_tester_helper.main_helper.get_failed_expectations_num).to_equal(0); + end; + procedure not_cont_join_excl_cols_as_lst is l_actual sys_refcursor; l_expected sys_refcursor; @@ -2399,6 +2474,19 @@ Diff:% ut.expect(ut3_tester_helper.main_helper.get_failed_expectations_num).to_equal(0); end; + procedure not_cont_join_excl_cols_as_lst2 is + l_actual sys_refcursor; + l_expected sys_refcursor; + begin + --Arrange + open l_actual for select rownum as rn, 'a' as "A_Column", 'c' as A_COLUMN, 'y' SOME_COL, 'd' "Some_Col" from dual a connect by level < 10; + open l_expected for select rownum * 20 as rn, 'a' as "A_Column", 'd' as A_COLUMN, 'x' SOME_COL, 'c' "Some_Col" from dual a connect by level < 4; + --Act + ut3_develop.ut.expect(l_actual).not_to(ut3_develop.contain(l_expected).exclude(ut3_develop.ut_varchar2_list('//Some_Col','A_COLUMN')).join_by('RN')); + --Assert + ut.expect(ut3_tester_helper.main_helper.get_failed_expectations_num).to_equal(0); + end; + procedure to_contain_duplicates is l_actual sys_refcursor; l_expected sys_refcursor; @@ -2440,6 +2528,16 @@ Diff:% ut.expect(l_actual_message).to_be_like(l_expected_message); end; + procedure to_not_contain_fails_1245 is + c1 sys_refcursor; + c2 sys_refcursor; + begin + open c1 for select 'a' as letter from dual union all select 'b' from dual; + open c2 for select 'c' as letter from dual; + ut3_develop.ut.expect(c1).not_to(ut3_develop.contain(c2)); + ut.expect(ut3_tester_helper.main_helper.get_failed_expectations_num).to_equal(0); + end; + procedure udt_messg_format_eq is l_actual sys_refcursor; l_expected sys_refcursor; diff --git a/test/ut3_user/expectations/test_expectations_cursor.pks b/test/ut3_user/expectations/test_expectations_cursor.pks index 0ed249a49..e7a289247 100644 --- a/test/ut3_user/expectations/test_expectations_cursor.pks +++ b/test/ut3_user/expectations/test_expectations_cursor.pks @@ -376,24 +376,42 @@ create or replace package test_expectations_cursor is --%test( Cursor not to contains data from another cursor) procedure cursor_not_to_contain; + --%test( Cursor not_to[contain] data from another cursor) + procedure cursor_not_to_contain2; + --%test( Cursor fail not to contains data from another cursor) procedure cursor_not_to_contain_fail; - + + --%test( Cursor fail not_to[contain] data from another cursor) + procedure cursor_not_to_contain_fail2; + --%test( Cursor not contains data from another cursor with joinby clause) procedure cursor_not_to_contain_joinby; + --%test( Cursor not_to[contain] data from another cursor with joinby clause) + procedure cursor_not_to_contain_joinby2; + --%test(Cursor not contains data with of columns to include and join by value) procedure not_cont_join_incl_cols_as_lst; + --%test(Cursor not_to[contain] data with of columns to include and join by value) + procedure not_cont_join_incl_cols_as_lst2; + --%test(Cursor not contains data with of columns to exclude and join by value) procedure not_cont_join_excl_cols_as_lst; + --%test(Cursor not_to[contain] data with of columns to exclude and join by value) + procedure not_cont_join_excl_cols_as_lst2; + --%test(Cursor to contain duplicates) procedure to_contain_duplicates; --%test(Cursor to contain duplicates fail) procedure to_contain_duplicates_fail; - + + --%test(Cursor using not_to[contain] fails #1245) + procedure to_not_contain_fails_1245; + --%test(Display a message with a uer defined type with only type name not structure on equal) procedure udt_messg_format_eq; From 0707b2d8a8cb44b77abe327c46b596c7899bbb9a Mon Sep 17 00:00:00 2001 From: Lukasz Wasylow Date: Tue, 14 Mar 2023 23:19:56 -0700 Subject: [PATCH 14/77] Fixing when test name is longer than 30 char --- test/ut3_user/expectations/test_expectations_cursor.pkb | 4 ++-- test/ut3_user/expectations/test_expectations_cursor.pks | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/test/ut3_user/expectations/test_expectations_cursor.pkb b/test/ut3_user/expectations/test_expectations_cursor.pkb index 7d0f308a8..847cce9a0 100644 --- a/test/ut3_user/expectations/test_expectations_cursor.pkb +++ b/test/ut3_user/expectations/test_expectations_cursor.pkb @@ -2448,7 +2448,7 @@ Diff:% ut.expect(ut3_tester_helper.main_helper.get_failed_expectations_num).to_equal(0); end; - procedure not_cont_join_incl_cols_as_lst2 is + procedure not_con_join_incl_cols_as_lst2 is l_actual sys_refcursor; l_expected sys_refcursor; begin @@ -2474,7 +2474,7 @@ Diff:% ut.expect(ut3_tester_helper.main_helper.get_failed_expectations_num).to_equal(0); end; - procedure not_cont_join_excl_cols_as_lst2 is + procedure not_con_join_excl_cols_as_lst2 is l_actual sys_refcursor; l_expected sys_refcursor; begin diff --git a/test/ut3_user/expectations/test_expectations_cursor.pks b/test/ut3_user/expectations/test_expectations_cursor.pks index e7a289247..0f34486a9 100644 --- a/test/ut3_user/expectations/test_expectations_cursor.pks +++ b/test/ut3_user/expectations/test_expectations_cursor.pks @@ -395,13 +395,13 @@ create or replace package test_expectations_cursor is procedure not_cont_join_incl_cols_as_lst; --%test(Cursor not_to[contain] data with of columns to include and join by value) - procedure not_cont_join_incl_cols_as_lst2; + procedure not_con_join_incl_cols_as_lst2; --%test(Cursor not contains data with of columns to exclude and join by value) procedure not_cont_join_excl_cols_as_lst; --%test(Cursor not_to[contain] data with of columns to exclude and join by value) - procedure not_cont_join_excl_cols_as_lst2; + procedure not_con_join_excl_cols_as_lst2; --%test(Cursor to contain duplicates) procedure to_contain_duplicates; From 8460a3092ac4f6781bf8f07f5a0d26d31ab6fa71 Mon Sep 17 00:00:00 2001 From: Lukasz Wasylow Date: Wed, 15 Mar 2023 00:24:05 -0700 Subject: [PATCH 15/77] Update test --- .../expectations/test_expectation_anydata.pkb | 19 ++++++++++++++++++- .../expectations/test_expectation_anydata.pks | 3 +++ 2 files changed, 21 insertions(+), 1 deletion(-) diff --git a/test/ut3_user/expectations/test_expectation_anydata.pkb b/test/ut3_user/expectations/test_expectation_anydata.pkb index c56c31930..c70d41f9e 100644 --- a/test/ut3_user/expectations/test_expectation_anydata.pkb +++ b/test/ut3_user/expectations/test_expectation_anydata.pkb @@ -957,6 +957,23 @@ Rows: [ 60 differences, showing first 20 ] ut.expect(ut3_tester_helper.main_helper.get_failed_expectations_num).to_equal(0); end; + procedure collection_not_to_contain is + l_actual ut3_tester_helper.test_dummy_object_list; + l_expected ut3_tester_helper.test_dummy_object_list; + begin + --Arrange + select ut3_tester_helper.test_dummy_object( rownum, 'Something2 '||rownum, rownum+100) + bulk collect into l_actual + from dual connect by level <=4; + select ut3_tester_helper.test_dummy_object( rownum, 'Something '||rownum, rownum) + bulk collect into l_expected + from dual connect by level <=2 + order by rownum desc; + --Act + ut3_develop.ut.expect(anydata.convertCollection(l_actual)).not_to_contain(anydata.convertCollection(l_expected)); + ut.expect(ut3_tester_helper.main_helper.get_failed_expectations_num).to_equal(0); + end; + procedure object_to_contain is begin --Arrange @@ -967,7 +984,7 @@ Rows: [ 60 differences, showing first 20 ] ut3_develop.ut.expect(g_test_actual).to_contain(g_test_expected); ut.expect(ut3_tester_helper.main_helper.get_failed_expectations_num).to_equal(0); end; - + procedure arr_empty_eq_arr_empty_unord is begin --Arrange diff --git a/test/ut3_user/expectations/test_expectation_anydata.pks b/test/ut3_user/expectations/test_expectation_anydata.pks index 81dea74eb..54c246bd3 100644 --- a/test/ut3_user/expectations/test_expectation_anydata.pks +++ b/test/ut3_user/expectations/test_expectation_anydata.pks @@ -195,6 +195,9 @@ create or replace package test_expectation_anydata is --%test( Success when anydata collection contains data from another anydata collection) procedure collection_to_contain; + --%test( Success when anydata collection not contains data from another anydata collection) + procedure collection_not_to_contain; + --%test( Success when anydata object contains data from another anydata) procedure object_to_contain; From 7597905bfe41bc1b3580919e670d00e76bd62f3d Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" Date: Sun, 19 Mar 2023 17:05:14 +0000 Subject: [PATCH 16/77] Updated project version after build [skip ci] --- docs/about/authors.md | 2 +- docs/about/license.md | 2 +- docs/about/project-details.md | 2 +- docs/about/support.md | 2 +- docs/index.md | 2 +- docs/userguide/advanced_data_comparison.md | 2 +- docs/userguide/annotations.md | 2 +- docs/userguide/best-practices.md | 2 +- docs/userguide/coverage.md | 2 +- docs/userguide/exception-reporting.md | 2 +- docs/userguide/expectations.md | 2 +- docs/userguide/getting-started.md | 2 +- docs/userguide/install.md | 2 +- docs/userguide/querying_suites.md | 2 +- docs/userguide/reporters.md | 2 +- docs/userguide/running-unit-tests.md | 2 +- docs/userguide/upgrade.md | 2 +- source/core/ut_utils.pks | 2 +- 18 files changed, 18 insertions(+), 18 deletions(-) diff --git a/docs/about/authors.md b/docs/about/authors.md index 9bd1a10c6..d132d2085 100644 --- a/docs/about/authors.md +++ b/docs/about/authors.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4085--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4094--develop-blue.svg) ### utPLSQL v3 Major Contributors diff --git a/docs/about/license.md b/docs/about/license.md index 727b513eb..21b82ee41 100644 --- a/docs/about/license.md +++ b/docs/about/license.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4085--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4094--develop-blue.svg) # Version Information diff --git a/docs/about/project-details.md b/docs/about/project-details.md index 5f4b1d57c..9b423d5a8 100644 --- a/docs/about/project-details.md +++ b/docs/about/project-details.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4085--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4094--develop-blue.svg) # utPLSQL Project Details diff --git a/docs/about/support.md b/docs/about/support.md index cd2f76d49..d534857fa 100644 --- a/docs/about/support.md +++ b/docs/about/support.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4085--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4094--develop-blue.svg) # How to get support diff --git a/docs/index.md b/docs/index.md index b029f6206..d6884ba2d 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4085--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4094--develop-blue.svg) ## What is utPLSQL diff --git a/docs/userguide/advanced_data_comparison.md b/docs/userguide/advanced_data_comparison.md index 7a2f511aa..2c31a9a89 100644 --- a/docs/userguide/advanced_data_comparison.md +++ b/docs/userguide/advanced_data_comparison.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4085--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4094--develop-blue.svg) # Advanced data comparison diff --git a/docs/userguide/annotations.md b/docs/userguide/annotations.md index 0e3b6446e..d193e2c20 100644 --- a/docs/userguide/annotations.md +++ b/docs/userguide/annotations.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4085--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4094--develop-blue.svg) Annotations are used to configure tests and suites in a declarative way similar to modern OOP languages. This way, test configuration is stored along with the test logic inside the test package. No additional configuration files or tables are needed for test cases. The annotation names are based on popular testing frameworks such as JUnit. diff --git a/docs/userguide/best-practices.md b/docs/userguide/best-practices.md index d41641303..63c0e36e9 100644 --- a/docs/userguide/best-practices.md +++ b/docs/userguide/best-practices.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4085--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4094--develop-blue.svg) The following are best practices we at utPLSQL have learned about PL/SQL and Unit Testing. diff --git a/docs/userguide/coverage.md b/docs/userguide/coverage.md index 5e1c4cbdb..f77717798 100644 --- a/docs/userguide/coverage.md +++ b/docs/userguide/coverage.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4085--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4094--develop-blue.svg) utPLSQL comes with a built-in coverage reporting engine. The code coverage reporting uses package DBMS_PROFILER (and DBMS_PLSQL_CODE_COVERAGE on Oracle database version 12.2 and above) provided with Oracle database. Code coverage is gathered for the following source types: diff --git a/docs/userguide/exception-reporting.md b/docs/userguide/exception-reporting.md index 8c986cb87..e237bf4ef 100644 --- a/docs/userguide/exception-reporting.md +++ b/docs/userguide/exception-reporting.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4085--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4094--develop-blue.svg) utPLSQL is responsible for handling exceptions wherever they occur in the test run. The framework is trapping most of the exceptions so that the test execution is not affected by individual tests or test packages throwing an exception. The framework provides a full stacktrace for every exception that was thrown. The reported stacktrace does not include any utPLSQL library calls in it. diff --git a/docs/userguide/expectations.md b/docs/userguide/expectations.md index c116464de..8fa525a69 100644 --- a/docs/userguide/expectations.md +++ b/docs/userguide/expectations.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4085--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4094--develop-blue.svg) ## Expectation concepts diff --git a/docs/userguide/getting-started.md b/docs/userguide/getting-started.md index 4427f2f10..d97d49464 100644 --- a/docs/userguide/getting-started.md +++ b/docs/userguide/getting-started.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4085--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4094--develop-blue.svg) # Getting started with TDD and utPLSQL diff --git a/docs/userguide/install.md b/docs/userguide/install.md index 7af30194d..934d07d72 100644 --- a/docs/userguide/install.md +++ b/docs/userguide/install.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4085--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4094--develop-blue.svg) ## Supported database versions diff --git a/docs/userguide/querying_suites.md b/docs/userguide/querying_suites.md index 68c689828..9e16fd3e8 100644 --- a/docs/userguide/querying_suites.md +++ b/docs/userguide/querying_suites.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4085--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4094--develop-blue.svg) ## Obtaining information about suites diff --git a/docs/userguide/reporters.md b/docs/userguide/reporters.md index 75bd4df3d..b3b0e338a 100644 --- a/docs/userguide/reporters.md +++ b/docs/userguide/reporters.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4085--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4094--develop-blue.svg) utPLSQL provides several reporting formats. The sections below describe most of them. diff --git a/docs/userguide/running-unit-tests.md b/docs/userguide/running-unit-tests.md index 07b192ce1..e8e431980 100644 --- a/docs/userguide/running-unit-tests.md +++ b/docs/userguide/running-unit-tests.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4085--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4094--develop-blue.svg) utPLSQL framework provides two main entry points to run unit tests from within the database: diff --git a/docs/userguide/upgrade.md b/docs/userguide/upgrade.md index bb66e6e0e..47772937e 100644 --- a/docs/userguide/upgrade.md +++ b/docs/userguide/upgrade.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4085--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4094--develop-blue.svg) # Upgrading from version 2 diff --git a/source/core/ut_utils.pks b/source/core/ut_utils.pks index 18a3d5952..f92fbc2b0 100644 --- a/source/core/ut_utils.pks +++ b/source/core/ut_utils.pks @@ -21,7 +21,7 @@ create or replace package ut_utils authid definer is * */ - gc_version constant varchar2(50) := 'v3.1.14.4085-develop'; + gc_version constant varchar2(50) := 'v3.1.14.4094-develop'; subtype t_executable_type is varchar2(30); gc_before_all constant t_executable_type := 'beforeall'; From 521fe472bb634095a3a3081f5175933eec61d2aa Mon Sep 17 00:00:00 2001 From: Jacek Gebal Date: Mon, 20 Mar 2023 00:20:15 +0200 Subject: [PATCH 17/77] Performance improvements in output buffer for reporters that only produce data after the whole run is finished. --- source/api/ut.pkb | 7 +- .../output_buffers/ut_output_bulk_buffer.tpb | 160 ++++++++++++++++++ .../output_buffers/ut_output_bulk_buffer.tps | 27 +++ source/core/ut_utils.pkb | 1 + source/install.sql | 2 + .../ut_coverage_cobertura_reporter.tpb | 2 +- .../reporters/ut_coverage_html_reporter.tpb | 2 +- .../reporters/ut_coverage_sonar_reporter.tpb | 9 +- source/reporters/ut_coveralls_reporter.tpb | 2 +- source/reporters/ut_junit_reporter.tpb | 2 +- source/reporters/ut_sonar_test_reporter.tpb | 2 +- source/reporters/ut_tfs_junit_reporter.tpb | 2 +- source/reporters/ut_xunit_reporter.tpb | 2 +- source/uninstall_objects.sql | 2 + test/ut3_tester_helper/coverage_helper.pkb | 2 +- 15 files changed, 214 insertions(+), 10 deletions(-) create mode 100644 source/core/output_buffers/ut_output_bulk_buffer.tpb create mode 100644 source/core/output_buffers/ut_output_bulk_buffer.tps diff --git a/source/api/ut.pkb b/source/api/ut.pkb index 5e612efb4..cff35b771 100644 --- a/source/api/ut.pkb +++ b/source/api/ut.pkb @@ -278,8 +278,8 @@ create or replace package body ut is ); if l_reporter is of (ut_output_reporter_base) then l_results := treat(l_reporter as ut_output_reporter_base).get_lines_cursor(); + g_result_lines := ut_varchar2_list(); loop - g_result_lines := ut_varchar2_list(); pipe row( get_report_outputs( l_results ) ); end loop; end if; @@ -326,6 +326,7 @@ create or replace package body ut is ); if l_reporter is of (ut_output_reporter_base) then l_results := treat(l_reporter as ut_output_reporter_base).get_lines_cursor(); + g_result_lines := ut_varchar2_list(); loop pipe row( get_report_outputs( l_results ) ); end loop; @@ -374,6 +375,7 @@ create or replace package body ut is ); if l_reporter is of (ut_output_reporter_base) then l_results := treat(l_reporter as ut_output_reporter_base).get_lines_cursor(); + g_result_lines := ut_varchar2_list(); loop pipe row( get_report_outputs( l_results ) ); end loop; @@ -422,6 +424,7 @@ create or replace package body ut is ); if l_reporter is of (ut_output_reporter_base) then l_results := treat(l_reporter as ut_output_reporter_base).get_lines_cursor(); + g_result_lines := ut_varchar2_list(); loop pipe row( get_report_outputs( l_results ) ); end loop; @@ -470,6 +473,7 @@ create or replace package body ut is ); if l_reporter is of (ut_output_reporter_base) then l_results := treat(l_reporter as ut_output_reporter_base).get_lines_cursor(); + g_result_lines := ut_varchar2_list(); loop pipe row( get_report_outputs( l_results ) ); end loop; @@ -518,6 +522,7 @@ create or replace package body ut is ); if l_reporter is of (ut_output_reporter_base) then l_results := treat(l_reporter as ut_output_reporter_base).get_lines_cursor(); + g_result_lines := ut_varchar2_list(); loop pipe row( get_report_outputs( l_results ) ); end loop; diff --git a/source/core/output_buffers/ut_output_bulk_buffer.tpb b/source/core/output_buffers/ut_output_bulk_buffer.tpb new file mode 100644 index 000000000..49aa1e913 --- /dev/null +++ b/source/core/output_buffers/ut_output_bulk_buffer.tpb @@ -0,0 +1,160 @@ +create or replace type body ut_output_bulk_buffer is + /* + utPLSQL - Version 3 + Copyright 2016 - 2023 utPLSQL Project + + Licensed under the Apache License, Version 2.0 (the "License"): + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + */ + + constructor function ut_output_bulk_buffer(self in out nocopy ut_output_bulk_buffer, a_output_id raw := null) return self as result is + begin + self.init(a_output_id, $$plsql_unit); + return; + end; + + overriding member procedure send_line(self in out nocopy ut_output_bulk_buffer, a_text varchar2, a_item_type varchar2 := null) is + pragma autonomous_transaction; + begin + if a_text is not null or a_item_type is not null then + if length(a_text) > ut_utils.gc_max_storage_varchar2_len then + self.send_lines( + ut_utils.convert_collection( + ut_utils.clob_to_table(a_text, ut_utils.gc_max_storage_varchar2_len) + ), + a_item_type + ); + else + self.last_write_message_id := self.last_write_message_id + 1; + insert /*+ no_parallel */ into ut_output_buffer_tmp(output_id, message_id, text, item_type) + values (self.output_id, self.last_write_message_id, a_text, a_item_type); + end if; + commit; + end if; + end; + + overriding member procedure send_lines(self in out nocopy ut_output_bulk_buffer, a_text_list ut_varchar2_rows, a_item_type varchar2 := null) is + pragma autonomous_transaction; + begin + insert /*+ no_parallel */ into ut_output_buffer_tmp(output_id, message_id, text, item_type) + select /*+ no_parallel */ self.output_id, self.last_write_message_id + rownum, t.column_value, a_item_type + from table(a_text_list) t + where t.column_value is not null or a_item_type is not null; + self.last_write_message_id := self.last_write_message_id + SQL%rowcount; + commit; + end; + + overriding member procedure send_clob(self in out nocopy ut_output_bulk_buffer, a_text clob, a_item_type varchar2 := null) is + pragma autonomous_transaction; + begin + if a_text is not null and a_text != empty_clob() or a_item_type is not null then + if length(a_text) > ut_utils.gc_max_storage_varchar2_len then + self.send_lines( + ut_utils.convert_collection( + ut_utils.clob_to_table(a_text, ut_utils.gc_max_storage_varchar2_len) + ), + a_item_type + ); + else + self.last_write_message_id := self.last_write_message_id + 1; + insert /*+ no_parallel */ into ut_output_buffer_tmp(output_id, message_id, text, item_type) + values (self.output_id, self.last_write_message_id, a_text, a_item_type); + end if; + commit; + end if; + end; + + overriding member procedure lines_to_dbms_output(self in ut_output_bulk_buffer, a_initial_timeout number := null, a_timeout_sec number := null) is + l_data sys_refcursor; + l_text ut_varchar2_rows; + l_item_type ut_varchar2_rows; + begin + l_data := self.get_lines_cursor(a_initial_timeout, a_timeout_sec); + loop + fetch l_data bulk collect into l_text, l_item_type limit 10000; + for idx in 1 .. l_text.count loop + dbms_output.put_line(l_text(idx)); + end loop; + exit when l_data%notfound; + end loop; + close l_data; + end; + + overriding member function get_lines_cursor(a_initial_timeout number := null, a_timeout_sec number := null) return sys_refcursor is + lc_init_wait_sec constant number := coalesce(a_initial_timeout, 30 ); + l_already_waited_sec number(10,2) := 0; + l_sleep_time number(2,1); + l_exists integer; + l_finished boolean := false; + l_data_produced boolean; + l_producer_active boolean := false; + l_producer_started boolean := false; + l_producer_finished boolean := false; + l_results sys_refcursor; + begin + + while not l_finished loop + + if not l_data_produced then + select /*+ no_parallel */ count(1) into l_exists + from ut_output_buffer_tmp o + where o.output_id = self.output_id and rownum = 1; + l_data_produced := (l_exists = 1); + end if; + + l_sleep_time := case when l_already_waited_sec >= 1 then 0.5 else 0.1 end; + l_producer_active := (self.get_lock_status() <> 0); + l_producer_started := (l_producer_active or l_data_produced ) or l_producer_started; + l_producer_finished := (l_producer_started and not l_producer_active) or l_producer_finished; + + l_finished := + self.timeout_producer_not_finished(l_producer_finished, l_already_waited_sec, a_timeout_sec) + or self.timeout_producer_not_started(l_producer_started, l_already_waited_sec, lc_init_wait_sec) + or l_producer_finished; + end loop; + + open l_results for + select /*+ no_parallel */ o.text, o.item_type + from ut_output_buffer_tmp o + where o.output_id = self.output_id + and o.text is not null + order by o.output_id, o.message_id; + + return l_results; + + end; + + /* Important note. + This function code is almost duplicated between two types for performance reasons. + The pipe row clause is much faster on VARCHAR2 then it is on clob. + That is the key reason for two implementations. + */ + overriding member function get_lines(a_initial_timeout number := null, a_timeout_sec number := null) return ut_output_data_rows pipelined is + l_data sys_refcursor; + l_text ut_varchar2_rows; + l_item_type ut_varchar2_rows; + begin + l_data := self.get_lines_cursor(a_initial_timeout, a_timeout_sec); + loop + fetch l_data bulk collect into l_text, l_item_type limit 10000; + for idx in 1 .. l_text.count loop + pipe row( ut_output_data_row(l_text(idx), l_item_type(idx)) ); + end loop; + exit when l_data%notfound; + end loop; + close l_data; + return; + self.remove_buffer_info(); + end; + +end; +/ diff --git a/source/core/output_buffers/ut_output_bulk_buffer.tps b/source/core/output_buffers/ut_output_bulk_buffer.tps new file mode 100644 index 000000000..d74d4ee14 --- /dev/null +++ b/source/core/output_buffers/ut_output_bulk_buffer.tps @@ -0,0 +1,27 @@ +create or replace type ut_output_bulk_buffer under ut_output_buffer_base ( + /* + utPLSQL - Version 3 + Copyright 2016 - 2023 utPLSQL Project + + Licensed under the Apache License, Version 2.0 (the "License"): + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + */ + + constructor function ut_output_bulk_buffer(self in out nocopy ut_output_bulk_buffer, a_output_id raw := null) return self as result, + overriding member procedure send_line(self in out nocopy ut_output_bulk_buffer, a_text varchar2, a_item_type varchar2 := null), + overriding member procedure send_lines(self in out nocopy ut_output_bulk_buffer, a_text_list ut_varchar2_rows, a_item_type varchar2 := null), + overriding member procedure send_clob(self in out nocopy ut_output_bulk_buffer, a_text clob, a_item_type varchar2 := null), + overriding member procedure lines_to_dbms_output(self in ut_output_bulk_buffer, a_initial_timeout number := null, a_timeout_sec number := null), + overriding member function get_lines_cursor(a_initial_timeout number := null, a_timeout_sec number := null) return sys_refcursor, + overriding member function get_lines(a_initial_timeout number := null, a_timeout_sec number := null) return ut_output_data_rows pipelined +) not final +/ diff --git a/source/core/ut_utils.pkb b/source/core/ut_utils.pkb index 162e50f8f..5114cecad 100644 --- a/source/core/ut_utils.pkb +++ b/source/core/ut_utils.pkb @@ -570,6 +570,7 @@ create or replace package body ut_utils is end loop; exit when l_lines_data%notfound; end loop; + close l_lines_data; execute immediate 'truncate table ut_dbms_output_cache'; commit; end; diff --git a/source/install.sql b/source/install.sql index 6f3c16801..0873d6a1e 100644 --- a/source/install.sql +++ b/source/install.sql @@ -120,6 +120,8 @@ create or replace context &&ut3_owner._info using &&ut3_owner..ut_session_contex @@install_component.sql 'core/output_buffers/ut_output_table_buffer.tpb' @@install_component.sql 'core/output_buffers/ut_output_clob_table_buffer.tps' @@install_component.sql 'core/output_buffers/ut_output_clob_table_buffer.tpb' +@@install_component.sql 'core/output_buffers/ut_output_bulk_buffer.tps' +@@install_component.sql 'core/output_buffers/ut_output_bulk_buffer.tpb' @@install_component.sql 'core/types/ut_output_reporter_base.tps' diff --git a/source/reporters/ut_coverage_cobertura_reporter.tpb b/source/reporters/ut_coverage_cobertura_reporter.tpb index 48082a6d4..50eb34631 100644 --- a/source/reporters/ut_coverage_cobertura_reporter.tpb +++ b/source/reporters/ut_coverage_cobertura_reporter.tpb @@ -20,7 +20,7 @@ create or replace type body ut_coverage_cobertura_reporter is self in out nocopy ut_coverage_cobertura_reporter ) return self as result is begin - self.init($$plsql_unit); + self.init($$plsql_unit,ut_output_bulk_buffer()); return; end; diff --git a/source/reporters/ut_coverage_html_reporter.tpb b/source/reporters/ut_coverage_html_reporter.tpb index c88c91a80..b1f9f6651 100644 --- a/source/reporters/ut_coverage_html_reporter.tpb +++ b/source/reporters/ut_coverage_html_reporter.tpb @@ -22,7 +22,7 @@ create or replace type body ut_coverage_html_reporter is a_html_report_assets_path varchar2 := null ) return self as result is begin - self.init($$plsql_unit); + self.init($$plsql_unit,ut_output_bulk_buffer()); self.project_name := a_project_name; assets_path := nvl(a_html_report_assets_path, ut_coverage_report_html_helper.get_default_html_assets_path()); return; diff --git a/source/reporters/ut_coverage_sonar_reporter.tpb b/source/reporters/ut_coverage_sonar_reporter.tpb index 8ee78dc24..1861a9be3 100644 --- a/source/reporters/ut_coverage_sonar_reporter.tpb +++ b/source/reporters/ut_coverage_sonar_reporter.tpb @@ -20,7 +20,7 @@ create or replace type body ut_coverage_sonar_reporter is self in out nocopy ut_coverage_sonar_reporter ) return self as result is begin - self.init($$plsql_unit); + self.init($$plsql_unit,ut_output_bulk_buffer()); return; end; @@ -84,6 +84,12 @@ create or replace type body ut_coverage_sonar_reporter is end; begin +-- execute immediate 'alter session set statistics_level=all'; +-- dbms_hprof.start_profiling( +-- location => 'PLSHPROF_DIR' +-- , filename => 'profiler_utPLSQL_run_on_'||$$plsql_unit||'_'||rawtohex(self.id)||'.txt' +-- ); +-- ut_coverage.coverage_stop(); self.print_text_lines( @@ -92,6 +98,7 @@ create or replace type body ut_coverage_sonar_reporter is a_run ) ); +-- dbms_hprof.stop_profiling; end; overriding member function get_description return varchar2 as diff --git a/source/reporters/ut_coveralls_reporter.tpb b/source/reporters/ut_coveralls_reporter.tpb index 3a54aa7f7..14672303e 100644 --- a/source/reporters/ut_coveralls_reporter.tpb +++ b/source/reporters/ut_coveralls_reporter.tpb @@ -20,7 +20,7 @@ create or replace type body ut_coveralls_reporter is self in out nocopy ut_coveralls_reporter ) return self as result is begin - self.init($$plsql_unit); + self.init($$plsql_unit,ut_output_bulk_buffer()); return; end; diff --git a/source/reporters/ut_junit_reporter.tpb b/source/reporters/ut_junit_reporter.tpb index 44affa030..3bceb52a4 100644 --- a/source/reporters/ut_junit_reporter.tpb +++ b/source/reporters/ut_junit_reporter.tpb @@ -18,7 +18,7 @@ create or replace type body ut_junit_reporter is constructor function ut_junit_reporter(self in out nocopy ut_junit_reporter) return self as result is begin - self.init($$plsql_unit); + self.init($$plsql_unit,ut_output_bulk_buffer()); return; end; diff --git a/source/reporters/ut_sonar_test_reporter.tpb b/source/reporters/ut_sonar_test_reporter.tpb index a3ec72241..7c46879d2 100644 --- a/source/reporters/ut_sonar_test_reporter.tpb +++ b/source/reporters/ut_sonar_test_reporter.tpb @@ -20,7 +20,7 @@ create or replace type body ut_sonar_test_reporter is self in out nocopy ut_sonar_test_reporter ) return self as result is begin - self.init($$plsql_unit); + self.init($$plsql_unit,ut_output_bulk_buffer()); return; end; diff --git a/source/reporters/ut_tfs_junit_reporter.tpb b/source/reporters/ut_tfs_junit_reporter.tpb index 710c3bc7f..e2d45a369 100644 --- a/source/reporters/ut_tfs_junit_reporter.tpb +++ b/source/reporters/ut_tfs_junit_reporter.tpb @@ -18,7 +18,7 @@ create or replace type body ut_tfs_junit_reporter is constructor function ut_tfs_junit_reporter(self in out nocopy ut_tfs_junit_reporter) return self as result is begin - self.init($$plsql_unit); + self.init($$plsql_unit,ut_output_bulk_buffer()); return; end; diff --git a/source/reporters/ut_xunit_reporter.tpb b/source/reporters/ut_xunit_reporter.tpb index eab0c8a3c..4612fb640 100644 --- a/source/reporters/ut_xunit_reporter.tpb +++ b/source/reporters/ut_xunit_reporter.tpb @@ -18,7 +18,7 @@ create or replace type body ut_xunit_reporter is constructor function ut_xunit_reporter(self in out nocopy ut_xunit_reporter) return self as result is begin - self.init($$plsql_unit); + self.init($$plsql_unit,ut_output_bulk_buffer()); return; end; diff --git a/source/uninstall_objects.sql b/source/uninstall_objects.sql index 571d187a2..9531736b9 100644 --- a/source/uninstall_objects.sql +++ b/source/uninstall_objects.sql @@ -323,6 +323,8 @@ drop type ut_output_table_buffer force; drop type ut_output_clob_table_buffer force; +drop type ut_output_bulk_buffer force; + drop type ut_output_buffer_base force; drop table ut_output_buffer_tmp purge; diff --git a/test/ut3_tester_helper/coverage_helper.pkb b/test/ut3_tester_helper/coverage_helper.pkb index d234967e4..ef14c586f 100644 --- a/test/ut3_tester_helper/coverage_helper.pkb +++ b/test/ut3_tester_helper/coverage_helper.pkb @@ -493,8 +493,8 @@ create or replace package body coverage_helper is x dbms_output.chararr; i integer := 100000; begin - execute immediate 'alter session set statistics_level=all'; /* + execute immediate 'alter session set statistics_level=all'; dbms_hprof.start_profiling( location => 'PLSHPROF_DIR' , filename => 'profiler_utPLSQL_run_]'||rawtohex(l_coverage_id)||q'[.txt' From 6c0b6e681954c389ca2763edcc6c56a7d142b586 Mon Sep 17 00:00:00 2001 From: Jacek Gebal Date: Mon, 20 Mar 2023 12:35:05 +0200 Subject: [PATCH 18/77] Fixes to new data-buffer implementation. --- .../core/output_buffers/ut_output_bulk_buffer.tpb | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/source/core/output_buffers/ut_output_bulk_buffer.tpb b/source/core/output_buffers/ut_output_bulk_buffer.tpb index 49aa1e913..ceae07862 100644 --- a/source/core/output_buffers/ut_output_bulk_buffer.tpb +++ b/source/core/output_buffers/ut_output_bulk_buffer.tpb @@ -49,7 +49,7 @@ create or replace type body ut_output_bulk_buffer is select /*+ no_parallel */ self.output_id, self.last_write_message_id + rownum, t.column_value, a_item_type from table(a_text_list) t where t.column_value is not null or a_item_type is not null; - self.last_write_message_id := self.last_write_message_id + SQL%rowcount; + self.last_write_message_id := self.last_write_message_id + sql%rowcount; commit; end; @@ -87,15 +87,16 @@ create or replace type body ut_output_bulk_buffer is exit when l_data%notfound; end loop; close l_data; + self.remove_buffer_info(); end; overriding member function get_lines_cursor(a_initial_timeout number := null, a_timeout_sec number := null) return sys_refcursor is - lc_init_wait_sec constant number := coalesce(a_initial_timeout, 30 ); + lc_init_wait_sec constant number := coalesce(a_initial_timeout, 10 ); l_already_waited_sec number(10,2) := 0; l_sleep_time number(2,1); l_exists integer; l_finished boolean := false; - l_data_produced boolean; + l_data_produced boolean := false; l_producer_active boolean := false; l_producer_started boolean := false; l_producer_finished boolean := false; @@ -115,11 +116,13 @@ create or replace type body ut_output_bulk_buffer is l_producer_active := (self.get_lock_status() <> 0); l_producer_started := (l_producer_active or l_data_produced ) or l_producer_started; l_producer_finished := (l_producer_started and not l_producer_active) or l_producer_finished; - l_finished := self.timeout_producer_not_finished(l_producer_finished, l_already_waited_sec, a_timeout_sec) or self.timeout_producer_not_started(l_producer_started, l_already_waited_sec, lc_init_wait_sec) or l_producer_finished; + + dbms_lock.sleep(l_sleep_time); + l_already_waited_sec := l_already_waited_sec + l_sleep_time; end loop; open l_results for @@ -152,8 +155,8 @@ create or replace type body ut_output_bulk_buffer is exit when l_data%notfound; end loop; close l_data; - return; self.remove_buffer_info(); + return; end; end; From 0daab3389dbf3fd0a0ec28e800479cfff486d45c Mon Sep 17 00:00:00 2001 From: Lukasz Wasylow Date: Mon, 27 Mar 2023 20:37:09 -0700 Subject: [PATCH 19/77] Checkpoint --- docs/userguide/annotations.md | 71 ++++++++++-- source/api/ut_runner.pkb | 14 +-- source/core/types/ut_run.tpb | 2 +- source/core/types/ut_run.tps | 6 +- source/core/ut_suite_builder.pkb | 2 +- source/core/ut_suite_cache_manager.pkb | 150 ++++++++++++++++++------- source/core/ut_suite_cache_manager.pks | 5 +- source/core/ut_suite_manager.pkb | 6 +- source/core/ut_suite_manager.pks | 2 +- source/core/ut_utils.pks | 4 + test/ut3_user/api/test_ut_run.pkb | 74 +++++++++++- test/ut3_user/api/test_ut_run.pks | 15 +++ 12 files changed, 280 insertions(+), 71 deletions(-) diff --git a/docs/userguide/annotations.md b/docs/userguide/annotations.md index 0e3b6446e..7fd2c09fa 100644 --- a/docs/userguide/annotations.md +++ b/docs/userguide/annotations.md @@ -1616,11 +1616,8 @@ or Tags are defined as a comma separated list within the `--%tags` annotation. -When executing a test run with tag filter applied, the framework will find all tests associated with the given tags and execute them. -The framework applies `OR` logic to all specified tags so any test / suite that matches at least one tag will be included in the test run. - -When a suite/context is tagged, all of its children will automatically inherit the tag and get executed along with the parent. Parent suite tests are not executed, but a suitepath hierarchy is kept. - +When a suite/context is tagged, all of its children will automatically inherit the tag and get executed along with the parent, unless they are excluded by tag expression. +Parent suite tests are not executed, but a suitepath hierarchy is kept. Sample test suite package with tags. ```sql linenums="1" @@ -1661,7 +1658,37 @@ end ut_sample_test; / ``` -Execution of the test is done by using the parameter `a_tags` +#### Tag Expressions + +Tag expressions are boolean expressions with the operators !, & and |. In addition, ( and ) can be used to adjust for operator precedence. + +Two special expressions are supported, any() and none(), which select all tests with any tags at all, and all tests without any tags, respectively. These special expressions may be combined with other expressions just like normal tags. + +| Operator | Meaning | +| -------- | --------| +| ! | not | +| & | and | +| \| | or | + +If you are tagging your tests across multiple dimensions, tag expressions help you to select which tests to execute. When tagging by test type (e.g., micro, integration, end-to-end) and feature (e.g., product, catalog, shipping), the following tag expressions can be useful. + + +| Tag Expression | Selection | +| -------- | --------| +| product | all tests for product | +| catalog \| shipping | all tests for catalog plus all tests for shipping | +| catalog & shipping | all tests for the intersection between catalog and shipping | +| product & !end-to-end | all tests for product, but not the end-to-end tests | +| (micro \| integration) & (product \| shipping) | all micro or integration tests for product or shipping | + + +Execution of the test is done by using the parameter `a_tags` with tag expressions + + +```sql linenums="1" +select * from table(ut.run(a_tags => 'fast||!complex')); +``` +The above call will execute all tests from `ut_sample_test` package as the whole suite is tagged with `api` because a suite meet expression condition. ```sql linenums="1" select * from table(ut.run(a_path => 'ut_sample_test',a_tags => 'api')); @@ -1683,8 +1710,14 @@ The above call will execute both `ut_sample_test.ut_refcursors1` and `ut_sample_ Tags must follow the below naming convention: - tag is case sensitive -- tag can contain special characters like `$#/\?-!` etc. -- tag cannot be an empty string +- tag must not contain any of the following reserved characters: + - comma (,) + - left or right parenthesis ((, )) + - ampersand (&) + - vertical bar (|) + - exclamation point (!) +- tag cannot be null or blank +- tag cannot contain whitespace - tag cannot start with a dash, e.g. `-some-stuff` is **not** a valid tag - tag cannot contain spaces, e.g. `test of batch`. To create a multi-word tag use underscores or dashes, e.g. `test_of_batch`, `test-of-batch` - leading and trailing spaces are ignored in tag name, e.g. `--%tags( tag1 , tag2 )` becomes `tag1` and `tag2` tag names @@ -1693,13 +1726,31 @@ Tags must follow the below naming convention: #### Excluding tests/suites by tags It is possible to exclude parts of test suites with tags. -In order to do so, prefix the tag name to exclude with a `-` (dash) sign when invoking the test run. - +In order to do so, prefix the tag name to exclude with a `!` (exclamation) sign when invoking the test run which is equivalent of `-` (dash) in legacy notation. Examples (based on above sample test suite) +```sql linenums="1" +select * from table(ut.run(a_tags => '(api|fast)&!complex')); +``` + +which is equivalent of legacy calling: + ```sql linenums="1" select * from table(ut.run(a_tags => 'api,fast,-complex')); ``` + +or + +```sql linenums="1" +select * from table(ut.run(a_tags => '(api|fast)&(!complex&!test1)')); +``` + +which is equivalent of legacy calling: + +```sql linenums="1" +select * from table(ut.run(a_tags => 'api,fast,-complex,-test1')); +``` + The above call will execute all suites/contexts/tests that are marked with any of tags `api` or `fast` except those suites/contexts/tests that are marked as `complex`. Given the above example package `ut_sample_test`, only `ut_sample_test.ut_test` will be executed. diff --git a/source/api/ut_runner.pkb b/source/api/ut_runner.pkb index b69a51a04..2760868e0 100644 --- a/source/api/ut_runner.pkb +++ b/source/api/ut_runner.pkb @@ -79,7 +79,8 @@ create or replace package body ut_runner is l_coverage_schema_names ut_varchar2_rows; l_paths ut_varchar2_list; l_random_test_order_seed positive; - l_tags ut_varchar2_rows := ut_varchar2_rows(); + l_tags varchar2(4000) := a_tags; + begin ut_event_manager.initialize(); if a_reporters is not empty then @@ -94,6 +95,11 @@ create or replace package body ut_runner is ut_event_manager.trigger_event(ut_event_manager.gc_initialize); ut_event_manager.trigger_event(ut_event_manager.gc_debug, ut_run_info()); + --Verify tag tag expression is valid + if regexp_like(l_tags,'[&|]{2,}|[!-]{2,}|[!-][&|]|[^-&|!]+[-!]|[-!|&][)]') + or (regexp_count(l_tags,'\(') <> regexp_count(l_tags,'\)')) then + raise_application_error(ut_utils.gc_invalid_tag_expression, 'Invalid Tag expression'); + end if; if a_random_test_order_seed is not null then l_random_test_order_seed := a_random_test_order_seed; elsif a_random_test_order then @@ -118,12 +124,6 @@ create or replace package body ut_runner is l_coverage_schema_names := ut_suite_manager.get_schema_names(l_paths); end if; - - if a_tags is not null then - l_tags := l_tags multiset union distinct ut_utils.convert_collection( - ut_utils.trim_list_elements(ut_utils.filter_list(ut_utils.string_to_table(a_tags,','),ut_utils.gc_word_no_space)) - ); - end if; l_run := ut_run( a_run_paths => l_paths, a_coverage_options => ut_coverage_options( diff --git a/source/core/types/ut_run.tpb b/source/core/types/ut_run.tpb index cdafb30fc..660c88791 100644 --- a/source/core/types/ut_run.tpb +++ b/source/core/types/ut_run.tpb @@ -24,7 +24,7 @@ create or replace type body ut_run as a_test_file_mappings ut_file_mappings := null, a_client_character_set varchar2 := null, a_random_test_order_seed positive := null, - a_run_tags ut_varchar2_rows := null + a_run_tags varchar2 := null ) return self as result is begin self.run_paths := a_run_paths; diff --git a/source/core/types/ut_run.tps b/source/core/types/ut_run.tps index debf847cd..8ce80d2a1 100644 --- a/source/core/types/ut_run.tps +++ b/source/core/types/ut_run.tps @@ -1,4 +1,4 @@ -create or replace type ut_run under ut_suite_item ( +create or replace type ut_run force under ut_suite_item ( /* utPLSQL - Version 3 Copyright 2016 - 2021 utPLSQL Project @@ -21,7 +21,7 @@ create or replace type ut_run under ut_suite_item ( project_name varchar2(4000), items ut_suite_items, run_paths ut_varchar2_list, - run_tags ut_varchar2_rows, + run_tags varchar2(4000), coverage_options ut_coverage_options, test_file_mappings ut_file_mappings, client_character_set varchar2(100), @@ -34,7 +34,7 @@ create or replace type ut_run under ut_suite_item ( a_test_file_mappings ut_file_mappings := null, a_client_character_set varchar2 := null, a_random_test_order_seed positive := null, - a_run_tags ut_varchar2_rows := null + a_run_tags varchar2 := null ) return self as result, overriding member procedure mark_as_skipped(self in out nocopy ut_run,a_skip_reason in varchar2), overriding member function do_execute(self in out nocopy ut_run) return boolean, diff --git a/source/core/ut_suite_builder.pkb b/source/core/ut_suite_builder.pkb index ebb113370..4aa2ff915 100644 --- a/source/core/ut_suite_builder.pkb +++ b/source/core/ut_suite_builder.pkb @@ -205,7 +205,7 @@ create or replace package body ut_suite_builder is l_tag_items := ut_utils.trim_list_elements(ut_utils.string_to_table(a_tags_ann_text(l_annotation_pos),',')); if l_tag_items is not empty then for i in 1 .. l_tag_items.count loop - if regexp_like(l_tag_items(i),'^[^-](\S)+$') then + if regexp_like(l_tag_items(i),'^[^-!&|](\S)+$') then l_tags_list.extend(); l_tags_list(l_tags_list.last) := l_tag_items(i); else diff --git a/source/core/ut_suite_cache_manager.pkb b/source/core/ut_suite_cache_manager.pkb index d3e832a9b..276f9427d 100644 --- a/source/core/ut_suite_cache_manager.pkb +++ b/source/core/ut_suite_cache_manager.pkb @@ -221,52 +221,118 @@ create or replace package body ut_suite_cache_manager is return l_suite_items; end; + /* + To support a legact tag notation + , = OR + - = NOT + we will perform a replace of that characters into + new notation. + || = OR + && = AND + ^ = NOT + */ + --TODO: How do we prevent when old notation reach 4k an new will be longer? + function replace_legacy_tag_notation(a_tags varchar2 + ) return varchar2 is + l_tags ut_varchar2_list := ut_utils.string_to_table(a_tags,','); + l_tags_include varchar2(2000); + l_tags_exclude varchar2(2000); + l_return_tag varchar2(4000); + begin + select listagg( t.column_value,' | ') + within group( order by column_value) + into l_tags_include + from table(l_tags) t + where t.column_value not like '-%'; + + select listagg( replace(t.column_value,'-','!'),' & ') + within group( order by column_value) + into l_tags_exclude + from table(l_tags) t + where t.column_value like '-%'; + + l_return_tag:= + case when l_tags_include is not null then + '('||l_tags_include||')' else null end || + case when l_tags_include is not null and l_tags_exclude is not null then + ' & ' else null end || + case when l_tags_exclude is not null then + '('||l_tags_exclude||')' else null end; + + return l_return_tag; + end; + + function create_where_filter(a_tags varchar2 + ) return varchar2 is + l_tags varchar2(4000):= replace(a_tags,' '); + begin + if instr(l_tags,',') > 0 or instr(l_tags,'-') > 0 then + l_tags := replace(replace_legacy_tag_notation(l_tags),' '); + end if; + l_tags := REGEXP_REPLACE(l_tags, + '(\(|\)|\||\!|\&)?([^|&!-]+)(\(|\)|\||\!|\&)?', + q'[\1q'<\2>' member of tags\3]'); + --replace operands to XPath + l_tags := REGEXP_REPLACE(l_tags, '\|',' or '); + l_tags := REGEXP_REPLACE(l_tags , '\&',' and '); + l_tags := REGEXP_REPLACE(l_tags,q'[(\!)(q'<[^|&!-]+>')( member of tags)]','\2 not \3'); + l_tags := '('||l_tags||')'; + return l_tags; + end; + /* Having a base set of suites we will do a further filter down if there are any tags defined. - */ + */ function get_tags_suites ( a_suite_items ut_suite_cache_rows, - a_tags ut_varchar2_rows + a_tags varchar2 ) return ut_suite_cache_rows is - l_suite_tags ut_suite_cache_rows := ut_suite_cache_rows(); - l_include_tags ut_varchar2_rows; - l_exclude_tags ut_varchar2_rows; + l_suite_tags ut_suite_cache_rows := ut_suite_cache_rows(); + l_sql varchar2(32000); + l_tags varchar2(4000):= create_where_filter(a_tags); begin - - select /*+ no_parallel */ column_value - bulk collect into l_include_tags - from table(a_tags) - where column_value not like '-%'; - - select /*+ no_parallel */ ltrim(column_value,'-') - bulk collect into l_exclude_tags - from table(a_tags) - where column_value like '-%'; - - with included_tags as ( - select c.path as path - from table(a_suite_items) c - where c.tags multiset intersect l_include_tags is not empty or l_include_tags is empty - ), - excluded_tags as ( - select c.path as path - from table(a_suite_items) c - where c.tags multiset intersect l_exclude_tags is not empty - ) - select value(c) as obj - bulk collect into l_suite_tags - from table(a_suite_items) c - where exists ( - select 1 from included_tags t - where t.path||'.' like c.path || '.%' /*all ancestors and self*/ - or c.path||'.' like t.path || '.%' /*all descendants and self*/ - ) - and not exists ( - select 1 from excluded_tags t - where c.path||'.' like t.path || '.%' /*all descendants and self*/ - ); - return l_suite_tags; + l_sql := + q'[ +with + suites_mv as ( + select c.id,value(c) as obj,c.path as path,c.self_type,c.object_owner,c.tags + from table(:suite_items) c + ), + suites_matching_expr as ( + select c.id,c.path as path,c.self_type,c.object_owner,c.tags + from suites_mv c + where c.self_type in ('UT_SUITE','UT_CONTEXT') + and ]'||l_tags||q'[ + ), + tests_matching_expr as ( + select c.id,c.path as path,c.self_type,c.object_owner,c.tags + from suites_mv c where c.self_type in ('UT_TEST') + and ]'||l_tags||q'[ + ), + tests_with_tags_inherited_from_suite as ( + select c.id,c.self_type,c.path,c.tags multiset union distinct t.tags tags,c.object_owner + from suites_mv c join suites_matching_expr t + on (c.path||'.' like t.path || '.%' /*all descendants and self*/ and c.object_owner = t.object_owner) + ), + tests_with_tags_promoted_to_suites as ( + select c.id,c.self_type,c.path,c.tags multiset union distinct t.tags tags,c.object_owner + from suites_mv c join tests_matching_expr t + on (t.path||'.' like c.path || '.%' /*all ancestors and self*/ and c.object_owner = t.object_owner) + ) + select obj from suites_mv c, + (select id,row_number() over (partition by id order by id) r_num from + (select id + from tests_with_tags_promoted_to_suites tst + where ]'||l_tags||q'[ + union all + select id from tests_with_tags_inherited_from_suite tst + where ]'||l_tags||q'[ + ) + ) t where c.id = t.id and r_num = 1 ]'; + + execute immediate l_sql bulk collect into l_suite_tags using a_suite_items; + return l_suite_tags; end; /* @@ -323,17 +389,17 @@ create or replace package body ut_suite_cache_manager is function get_cached_suite_rows( a_schema_paths ut_path_items, a_random_seed positive := null, - a_tags ut_varchar2_rows := null + a_tags varchar2 := null ) return ut_suite_cache_rows is l_results ut_suite_cache_rows := ut_suite_cache_rows(); l_suite_items ut_suite_cache_rows := ut_suite_cache_rows(); l_schema_paths ut_path_items; - l_tags ut_varchar2_rows := coalesce(a_tags,ut_varchar2_rows()); + l_tags varchar2(4000) := a_tags; begin l_schema_paths := a_schema_paths; l_suite_items := get_suite_items(a_schema_paths); - if l_tags.count > 0 then + if length(l_tags) > 0 then l_suite_items := get_tags_suites(l_suite_items,l_tags); end if; diff --git a/source/core/ut_suite_cache_manager.pks b/source/core/ut_suite_cache_manager.pks index 974babca5..72dd08800 100644 --- a/source/core/ut_suite_cache_manager.pks +++ b/source/core/ut_suite_cache_manager.pks @@ -57,7 +57,7 @@ create or replace package ut_suite_cache_manager authid definer is function get_cached_suite_rows( a_schema_paths ut_path_items, a_random_seed positive := null, - a_tags ut_varchar2_rows := null + a_tags varchar2 := null ) return ut_suite_cache_rows; function get_schema_paths(a_paths in ut_varchar2_list) return ut_path_items; @@ -95,5 +95,8 @@ create or replace package ut_suite_cache_manager authid definer is a_procedure_name varchar2 ) return boolean; + +function create_where_filter(a_tags varchar2 + ) return varchar2; end ut_suite_cache_manager; / diff --git a/source/core/ut_suite_manager.pkb b/source/core/ut_suite_manager.pkb index 86f68c076..0f8629ded 100644 --- a/source/core/ut_suite_manager.pkb +++ b/source/core/ut_suite_manager.pkb @@ -352,7 +352,7 @@ create or replace package body ut_suite_manager is function get_cached_suite_data( a_schema_paths ut_path_items, a_random_seed positive, - a_tags ut_varchar2_rows := null, + a_tags varchar2 := null, a_skip_all_objects boolean := false ) return t_cached_suites_cursor is l_unfiltered_rows ut_suite_cache_rows; @@ -451,7 +451,7 @@ create or replace package body ut_suite_manager is a_schema_paths ut_path_items, a_suites in out nocopy ut_suite_items, a_random_seed positive, - a_tags ut_varchar2_rows := null + a_tags varchar2 := null ) is begin reconstruct_from_cache( @@ -528,7 +528,7 @@ create or replace package body ut_suite_manager is a_paths ut_varchar2_list, a_suites out nocopy ut_suite_items, a_random_seed positive := null, - a_tags ut_varchar2_rows := ut_varchar2_rows() + a_tags varchar2 := null ) is l_paths ut_varchar2_list := a_paths; l_schema_names ut_varchar2_rows; diff --git a/source/core/ut_suite_manager.pks b/source/core/ut_suite_manager.pks index 65b3c0654..07539139a 100644 --- a/source/core/ut_suite_manager.pks +++ b/source/core/ut_suite_manager.pks @@ -53,7 +53,7 @@ create or replace package ut_suite_manager authid current_user is a_paths in ut_varchar2_list, a_suites out nocopy ut_suite_items, a_random_seed in positive := null, - a_tags ut_varchar2_rows := ut_varchar2_rows() + a_tags in varchar2 := null ); /** diff --git a/source/core/ut_utils.pks b/source/core/ut_utils.pks index 18a3d5952..f7f7b4f00 100644 --- a/source/core/ut_utils.pks +++ b/source/core/ut_utils.pks @@ -121,6 +121,10 @@ create or replace package ut_utils authid definer is ex_failed_open_cur exception; gc_failed_open_cur constant pls_integer := -20218; pragma exception_init (ex_failed_open_cur, -20218); + + ex_invalid_tag_expression exception; + gc_invalid_tag_expression constant pls_integer := -20219; + pragma exception_init (ex_invalid_tag_expression, -20219); gc_max_storage_varchar2_len constant integer := 4000; gc_max_output_string_length constant integer := 4000; diff --git a/test/ut3_user/api/test_ut_run.pkb b/test/ut3_user/api/test_ut_run.pkb index 41b969359..6f6e229c8 100644 --- a/test/ut3_user/api/test_ut_run.pkb +++ b/test/ut3_user/api/test_ut_run.pkb @@ -948,7 +948,7 @@ Failures:% procedure two_test_run_by_two_tags is l_results clob; begin - ut3_tester_helper.run_helper.run(a_tags => 'subtest1,subtest2'); + ut3_tester_helper.run_helper.run(a_tags => 'subtest1|subtest2'); l_results := ut3_tester_helper.main_helper.get_dbms_output_as_clob(); --Assert ut.expect( l_results ).to_be_like( '%test_package_1%' ); @@ -958,7 +958,21 @@ Failures:% ut.expect( l_results ).not_to_be_like( '%test_package_3%' ); ut.expect( l_results ).not_to_be_like( '%test_package_3%' ); end; - + + procedure two_test_run_by_two_tags_leg is + l_results clob; + begin + ut3_tester_helper.run_helper.run(a_tags => 'subtest1,subtest2'); + l_results := ut3_tester_helper.main_helper.get_dbms_output_as_clob(); + --Assert + ut.expect( l_results ).to_be_like( '%test_package_1%' ); + ut.expect( l_results ).to_be_like( '%test_package_2%' ); + ut.expect( l_results ).not_to_be_like( '%test_package_1.test2%' ); + ut.expect( l_results ).not_to_be_like( '%test_package_2.test2%' ); + ut.expect( l_results ).not_to_be_like( '%test_package_3%' ); + ut.expect( l_results ).not_to_be_like( '%test_package_3%' ); + end; + procedure suite_with_children_tag is l_results clob; begin @@ -1078,6 +1092,20 @@ Failures:% procedure tag_run_func_path_list is l_results ut3_develop.ut_varchar2_list; + begin + l_results := ut3_tester_helper.run_helper.run(ut3_develop.ut_varchar2_list(':tests.test_package_1',':tests'),a_tags => 'suite1test1|suite2test1'); + --Assert + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_be_like( '%test_package_1%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_be_like( '%test_package_2%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_be_like( '%test_package_1.test1%executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like( '%test_package_1.test2%executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_be_like( '%test_package_2.test1%executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like( '%test_package_2.test2%executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like( '%test_package_3%' ); + end; + + procedure tag_run_func_path_list_leg is + l_results ut3_develop.ut_varchar2_list; begin l_results := ut3_tester_helper.run_helper.run(ut3_develop.ut_varchar2_list(':tests.test_package_1',':tests'),a_tags => 'suite1test1,suite2test1'); --Assert @@ -1092,6 +1120,18 @@ Failures:% procedure tag_inc_exc_run_func_path_list is l_results ut3_develop.ut_varchar2_list; + begin + l_results := ut3_tester_helper.run_helper.run(ut3_develop.ut_varchar2_list(':tests.test_package_1',':tests'),a_tags => '(suite1test1|suite2test1)&!suite2'); + --Assert + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_be_like( '%test_package_1%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like( '%test_package_2%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_be_like( '%test_package_1.test1%executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like( '%test_package_1.test2%executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like( '%test_package_3%' ); + end; + + procedure tag_inc_exc_run_fun_pth_lst_lg is + l_results ut3_develop.ut_varchar2_list; begin l_results := ut3_tester_helper.run_helper.run(ut3_develop.ut_varchar2_list(':tests.test_package_1',':tests'),a_tags => 'suite1test1,suite2test1,-suite2'); --Assert @@ -1104,6 +1144,22 @@ Failures:% procedure tag_exclude_run_func_path_list is l_results ut3_develop.ut_varchar2_list; + begin + l_results := ut3_tester_helper.run_helper.run(ut3_develop.ut_varchar2_list(':tests,:tests2'),a_tags => '!suite1test2&!suite2test1&!test1suite3'); + --Assert + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_be_like( '%test_package_1%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_be_like( '%test_package_2%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_be_like( '%test_package_3%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_be_like( '%test_package_1.test1%executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like( '%test_package_1.test2%executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like( '%test_package_2.test1%executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_be_like( '%test_package_2.test2%executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like( '%test_package_3.test1%executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_be_like( '%test_package_3.test2%executed%' ); + end; + +procedure tag_exclude_run_fun_pth_lst_lg is + l_results ut3_develop.ut_varchar2_list; begin l_results := ut3_tester_helper.run_helper.run(ut3_develop.ut_varchar2_list(':tests,:tests2'),a_tags => '-suite1test2,-suite2test1,-test1suite3'); --Assert @@ -1120,6 +1176,20 @@ Failures:% procedure tag_include_exclude_run_func is l_results ut3_develop.ut_varchar2_list; + begin + l_results := ut3_tester_helper.run_helper.run(a_tags => '(suite1)&(!suite1test2&!suite2test1&!test1suite3)'); + --Assert + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_be_like( '%test_package_1%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_be_like( '%test_package_2%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like( '%test_package_3%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_be_like( '%test_package_1.test1%executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like( '%test_package_1.test2%executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like( '%test_package_2.test1%executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_be_like( '%test_package_2.test2%executed%' ); + end; + + procedure tag_include_exclude_run_fun_lg is + l_results ut3_develop.ut_varchar2_list; begin l_results := ut3_tester_helper.run_helper.run(a_tags => 'suite1,-suite1test2,-suite2test1,-test1suite3'); --Assert diff --git a/test/ut3_user/api/test_ut_run.pks b/test/ut3_user/api/test_ut_run.pks index b3be8700d..db02ee8f8 100644 --- a/test/ut3_user/api/test_ut_run.pks +++ b/test/ut3_user/api/test_ut_run.pks @@ -202,6 +202,9 @@ create or replace package test_ut_run is --%test(Execute tests by passing two tags) procedure two_test_run_by_two_tags; + --%test(Execute tests by passing two tags - Legacy notation) + procedure two_test_run_by_two_tags_leg; + --%test(Execute suite and all of its children) procedure suite_with_children_tag; @@ -235,15 +238,27 @@ create or replace package test_ut_run is --%test(Runs tests from given paths with paths list and a tag) procedure tag_run_func_path_list; + --%test(Runs tests from given paths with paths list and a tag - Legacy Notation) + procedure tag_run_func_path_list_leg; + --%test(Runs tests from given paths with paths list and include/exclude tags) procedure tag_inc_exc_run_func_path_list; + --%test(Runs tests from given paths with paths list and include/exclude tags - Legacy Notation) + procedure tag_inc_exc_run_fun_pth_lst_lg; + --%test(Runs tests from given path and excludes specific tags) procedure tag_exclude_run_func_path_list; + --%test(Runs tests from given path and excludes specific tags - Legacy Notation) + procedure tag_exclude_run_fun_pth_lst_lg; + --%test(Runs tests from given tags and exclude tags) procedure tag_include_exclude_run_func; + --%test(Runs tests from given tags and exclude tags - Legacy Notation) + procedure tag_include_exclude_run_fun_lg; + --%endcontext --%context(ut3_info context) From adbc76eb027f8425975b8277d4a10e2ec9ce7f7c Mon Sep 17 00:00:00 2001 From: Lukasz Wasylow Date: Mon, 27 Mar 2023 23:04:34 -0700 Subject: [PATCH 20/77] Address too long identified in 11g. Address flaky expression checker to be reworked. --- source/api/ut_runner.pkb | 6 ++++-- source/core/ut_suite_cache_manager.pkb | 10 +++++----- source/core/ut_suite_cache_manager.pks | 5 +---- 3 files changed, 10 insertions(+), 11 deletions(-) diff --git a/source/api/ut_runner.pkb b/source/api/ut_runner.pkb index 2760868e0..94538eb69 100644 --- a/source/api/ut_runner.pkb +++ b/source/api/ut_runner.pkb @@ -95,11 +95,13 @@ create or replace package body ut_runner is ut_event_manager.trigger_event(ut_event_manager.gc_initialize); ut_event_manager.trigger_event(ut_event_manager.gc_debug, ut_run_info()); - --Verify tag tag expression is valid - if regexp_like(l_tags,'[&|]{2,}|[!-]{2,}|[!-][&|]|[^-&|!]+[-!]|[-!|&][)]') + --TODO:Verify tag tag expression is valid + /* + if regexp_like(l_tags,'[&|]{2,}|[!-]{2,}|[!-][&|]|[^-&|!,]+[-!]|[-!|&][)]') or (regexp_count(l_tags,'\(') <> regexp_count(l_tags,'\)')) then raise_application_error(ut_utils.gc_invalid_tag_expression, 'Invalid Tag expression'); end if; + */ if a_random_test_order_seed is not null then l_random_test_order_seed := a_random_test_order_seed; elsif a_random_test_order then diff --git a/source/core/ut_suite_cache_manager.pkb b/source/core/ut_suite_cache_manager.pkb index 276f9427d..ff18deabf 100644 --- a/source/core/ut_suite_cache_manager.pkb +++ b/source/core/ut_suite_cache_manager.pkb @@ -270,7 +270,7 @@ create or replace package body ut_suite_cache_manager is l_tags := replace(replace_legacy_tag_notation(l_tags),' '); end if; l_tags := REGEXP_REPLACE(l_tags, - '(\(|\)|\||\!|\&)?([^|&!-]+)(\(|\)|\||\!|\&)?', + '(\(|\)|\||\!|\&)?([^|&!-()]+)(\(|\)|\||\!|\&)?', q'[\1q'<\2>' member of tags\3]'); --replace operands to XPath l_tags := REGEXP_REPLACE(l_tags, '\|',' or '); @@ -310,12 +310,12 @@ with from suites_mv c where c.self_type in ('UT_TEST') and ]'||l_tags||q'[ ), - tests_with_tags_inherited_from_suite as ( + tests_with_tags_inh_from_suite as ( select c.id,c.self_type,c.path,c.tags multiset union distinct t.tags tags,c.object_owner from suites_mv c join suites_matching_expr t on (c.path||'.' like t.path || '.%' /*all descendants and self*/ and c.object_owner = t.object_owner) ), - tests_with_tags_promoted_to_suites as ( + tests_with_tags_prom_to_suite as ( select c.id,c.self_type,c.path,c.tags multiset union distinct t.tags tags,c.object_owner from suites_mv c join tests_matching_expr t on (t.path||'.' like c.path || '.%' /*all ancestors and self*/ and c.object_owner = t.object_owner) @@ -323,10 +323,10 @@ with select obj from suites_mv c, (select id,row_number() over (partition by id order by id) r_num from (select id - from tests_with_tags_promoted_to_suites tst + from tests_with_tags_prom_to_suite tst where ]'||l_tags||q'[ union all - select id from tests_with_tags_inherited_from_suite tst + select id from tests_with_tags_inh_from_suite tst where ]'||l_tags||q'[ ) ) t where c.id = t.id and r_num = 1 ]'; diff --git a/source/core/ut_suite_cache_manager.pks b/source/core/ut_suite_cache_manager.pks index 72dd08800..7f06e95eb 100644 --- a/source/core/ut_suite_cache_manager.pks +++ b/source/core/ut_suite_cache_manager.pks @@ -94,9 +94,6 @@ create or replace package ut_suite_cache_manager authid definer is a_package_name varchar2, a_procedure_name varchar2 ) return boolean; - - -function create_where_filter(a_tags varchar2 - ) return varchar2; + end ut_suite_cache_manager; / From 1478b0d21035398f3b6eccf82b37fa91adeb0e6b Mon Sep 17 00:00:00 2001 From: Lukasz Wasylow Date: Wed, 29 Mar 2023 12:41:32 -0700 Subject: [PATCH 21/77] Adding validation for tag expression --- source/api/ut_runner.pkb | 7 +-- source/core/ut_utils.pkb | 63 +++++++++++++++++++++++++- source/core/ut_utils.pks | 2 + test/ut3_tester/core/test_ut_utils.pkb | 27 +++++++++++ test/ut3_tester/core/test_ut_utils.pks | 2 + 5 files changed, 95 insertions(+), 6 deletions(-) diff --git a/source/api/ut_runner.pkb b/source/api/ut_runner.pkb index 94538eb69..1ecaf1a42 100644 --- a/source/api/ut_runner.pkb +++ b/source/api/ut_runner.pkb @@ -95,13 +95,10 @@ create or replace package body ut_runner is ut_event_manager.trigger_event(ut_event_manager.gc_initialize); ut_event_manager.trigger_event(ut_event_manager.gc_debug, ut_run_info()); - --TODO:Verify tag tag expression is valid - /* - if regexp_like(l_tags,'[&|]{2,}|[!-]{2,}|[!-][&|]|[^-&|!,]+[-!]|[-!|&][)]') - or (regexp_count(l_tags,'\(') <> regexp_count(l_tags,'\)')) then + if ut_utils.valid_tag_expression(l_tags) = 0 then raise_application_error(ut_utils.gc_invalid_tag_expression, 'Invalid Tag expression'); end if; - */ + if a_random_test_order_seed is not null then l_random_test_order_seed := a_random_test_order_seed; elsif a_random_test_order then diff --git a/source/core/ut_utils.pkb b/source/core/ut_utils.pkb index 162e50f8f..03ce1ad55 100644 --- a/source/core/ut_utils.pkb +++ b/source/core/ut_utils.pkb @@ -988,7 +988,68 @@ create or replace package body ut_utils is return l_result; end; - + + function valid_tag_expression(a_tags in varchar2) return number is + t_left_side ut_varchar2_list := ut_varchar2_list('|','&',','); + t_right_side ut_varchar2_list := ut_varchar2_list('!','-'); + l_left_side_expression varchar2(100) := '[|&,]'; + l_left_side_regex varchar(400) := '([^|&,]*)[|&,](.*)'; + l_left_side varchar2(4000); + + l_rigth_side_expression varchar2(100) := '[!-]'; + l_right_side_regex varchar(400) := '([!-])([^!-].*)'; + l_right_side varchar2(4000); + + l_tags varchar2(4000) := a_tags; + l_result number :=1; + begin + --Validate that we have closed up all brackets + if regexp_count(l_tags,'\(') <> regexp_count(l_tags,'\)') then + l_result := 0; + end if; + + --Remove brackets as we dont evaluate expression only validate. + l_tags := replace(replace(l_tags,'('),')'); + + --Check if there are any left side operators for first in order from left to right + if regexp_count(l_tags,l_left_side_expression) > 0 then + --Extract left part of operator and remaining of string to right + l_left_side := regexp_replace(l_tags,l_left_side_regex,'\1'); + l_right_side := regexp_replace(l_tags,l_left_side_regex,'\2'); + + --If left side is null that means that we used left side operator without + -- left and right e.g. &test + if l_left_side is null then + l_result := 0; + else + --Extract right side from left side expression if there is any !- + --Remove first negation tag to see if there is double negation + l_left_side := regexp_replace(l_left_side,l_right_side_regex,'\2'); + end if; + + + --check that on right side there is no extra negation + if regexp_count(l_left_side,l_rigth_side_expression) > 0 then + l_result := 0; + end if; + + --Now process right side of string + if l_right_side is not null then + l_result := least(l_result,valid_tag_expression(l_right_side)); + else + l_result := 0; + end if; + else + --We just process single tag. + l_left_side := l_tags; + l_left_side := regexp_replace(l_left_side,l_right_side_regex,'\2'); + if regexp_count(l_left_side,l_rigth_side_expression) > 0 then + l_result := 0; + end if; + end if; + + return l_result; + end; end ut_utils; / diff --git a/source/core/ut_utils.pks b/source/core/ut_utils.pks index f7f7b4f00..2975c3846 100644 --- a/source/core/ut_utils.pks +++ b/source/core/ut_utils.pks @@ -477,5 +477,7 @@ create or replace package ut_utils authid definer is */ function interval_to_text(a_interval yminterval_unconstrained) return varchar2; + function valid_tag_expression(a_tags in varchar2) return number; + end ut_utils; / diff --git a/test/ut3_tester/core/test_ut_utils.pkb b/test/ut3_tester/core/test_ut_utils.pkb index 4ed718777..52c9cdff8 100644 --- a/test/ut3_tester/core/test_ut_utils.pkb +++ b/test/ut3_tester/core/test_ut_utils.pkb @@ -489,5 +489,32 @@ end; ut.expect(l_expected).to_equal(l_actual); end; + procedure valid_tag_expressions is + begin + ut.expect(1).to_equal(ut3_develop.ut_utils.valid_tag_expression('tag1')); + ut.expect(1).to_equal(ut3_develop.ut_utils.valid_tag_expression('tag1|tag2')); + ut.expect(1).to_equal(ut3_develop.ut_utils.valid_tag_expression('tag1&tag2')); + ut.expect(1).to_equal(ut3_develop.ut_utils.valid_tag_expression('!tag1')); + ut.expect(1).to_equal(ut3_develop.ut_utils.valid_tag_expression('tag1|!tag2')); + ut.expect(1).to_equal(ut3_develop.ut_utils.valid_tag_expression('tag1&!tag2')); + ut.expect(1).to_equal(ut3_develop.ut_utils.valid_tag_expression('!tag1|!tag2')); + ut.expect(1).to_equal(ut3_develop.ut_utils.valid_tag_expression('!tag1&!tag2')); + + ut.expect(1).to_equal(ut3_develop.ut_utils.valid_tag_expression('tag1,tag2')); + ut.expect(1).to_equal(ut3_develop.ut_utils.valid_tag_expression('-tag1')); + ut.expect(1).to_equal(ut3_develop.ut_utils.valid_tag_expression('tag1,-tag2')); + ut.expect(1).to_equal(ut3_develop.ut_utils.valid_tag_expression('-tag1,-tag2')); + + ut.expect(1).to_equal(ut3_develop.ut_utils.valid_tag_expression('(!tag1|!tag2)|tag3')); + ut.expect(1).to_equal(ut3_develop.ut_utils.valid_tag_expression('(!tag1&!tag2)|(tag3&tag4)')); + + ut.expect(0).to_equal(ut3_develop.ut_utils.valid_tag_expression('tag1|')); + ut.expect(0).to_equal(ut3_develop.ut_utils.valid_tag_expression('&!tag2')); + ut.expect(0).to_equal(ut3_develop.ut_utils.valid_tag_expression('!!tag1|!tag2')); + ut.expect(0).to_equal(ut3_develop.ut_utils.valid_tag_expression('!tag1&!tag2|')); + ut.expect(0).to_equal(ut3_develop.ut_utils.valid_tag_expression('((!tag1|!tag2)|tag3')); + + end; + end test_ut_utils; / diff --git a/test/ut3_tester/core/test_ut_utils.pks b/test/ut3_tester/core/test_ut_utils.pks index 4d83b5042..b87d3b2c6 100644 --- a/test/ut3_tester/core/test_ut_utils.pks +++ b/test/ut3_tester/core/test_ut_utils.pks @@ -154,6 +154,8 @@ create or replace package test_ut_utils is --%test(returns text representation of interval year to month for custom interval) procedure int_conv_ym_date; + --%test(Test to validate different type of expressions passed as tags) + procedure valid_tag_expressions; --%endcontext From b5ad7475b53f5b4b14d9805f9c56b8e36dde0883 Mon Sep 17 00:00:00 2001 From: Lukasz Wasylow Date: Wed, 29 Mar 2023 14:53:24 -0700 Subject: [PATCH 22/77] Comment out to see why its failing. --- source/api/ut_runner.pkb | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/source/api/ut_runner.pkb b/source/api/ut_runner.pkb index 1ecaf1a42..5f4912a24 100644 --- a/source/api/ut_runner.pkb +++ b/source/api/ut_runner.pkb @@ -94,10 +94,12 @@ create or replace package body ut_runner is ut_event_manager.trigger_event(ut_event_manager.gc_initialize); ut_event_manager.trigger_event(ut_event_manager.gc_debug, ut_run_info()); - + + /* if ut_utils.valid_tag_expression(l_tags) = 0 then raise_application_error(ut_utils.gc_invalid_tag_expression, 'Invalid Tag expression'); end if; + */ if a_random_test_order_seed is not null then l_random_test_order_seed := a_random_test_order_seed; From 06cb054aa72b8aca8115f277e673485321efc0ad Mon Sep 17 00:00:00 2001 From: Lukasz Wasylow Date: Wed, 29 Mar 2023 15:10:34 -0700 Subject: [PATCH 23/77] Revert "Comment out to see why its failing." This reverts commit b5ad7475b53f5b4b14d9805f9c56b8e36dde0883. Revert "Adding validation for tag expression" This reverts commit 1478b0d21035398f3b6eccf82b37fa91adeb0e6b. --- source/api/ut_runner.pkb | 7 +-- source/core/ut_utils.pkb | 63 +------------------------- source/core/ut_utils.pks | 2 - test/ut3_tester/core/test_ut_utils.pkb | 27 ----------- test/ut3_tester/core/test_ut_utils.pks | 2 - 5 files changed, 5 insertions(+), 96 deletions(-) diff --git a/source/api/ut_runner.pkb b/source/api/ut_runner.pkb index 5f4912a24..94538eb69 100644 --- a/source/api/ut_runner.pkb +++ b/source/api/ut_runner.pkb @@ -94,13 +94,14 @@ create or replace package body ut_runner is ut_event_manager.trigger_event(ut_event_manager.gc_initialize); ut_event_manager.trigger_event(ut_event_manager.gc_debug, ut_run_info()); - + + --TODO:Verify tag tag expression is valid /* - if ut_utils.valid_tag_expression(l_tags) = 0 then + if regexp_like(l_tags,'[&|]{2,}|[!-]{2,}|[!-][&|]|[^-&|!,]+[-!]|[-!|&][)]') + or (regexp_count(l_tags,'\(') <> regexp_count(l_tags,'\)')) then raise_application_error(ut_utils.gc_invalid_tag_expression, 'Invalid Tag expression'); end if; */ - if a_random_test_order_seed is not null then l_random_test_order_seed := a_random_test_order_seed; elsif a_random_test_order then diff --git a/source/core/ut_utils.pkb b/source/core/ut_utils.pkb index 03ce1ad55..162e50f8f 100644 --- a/source/core/ut_utils.pkb +++ b/source/core/ut_utils.pkb @@ -988,68 +988,7 @@ create or replace package body ut_utils is return l_result; end; - - function valid_tag_expression(a_tags in varchar2) return number is - t_left_side ut_varchar2_list := ut_varchar2_list('|','&',','); - t_right_side ut_varchar2_list := ut_varchar2_list('!','-'); - l_left_side_expression varchar2(100) := '[|&,]'; - l_left_side_regex varchar(400) := '([^|&,]*)[|&,](.*)'; - l_left_side varchar2(4000); - - l_rigth_side_expression varchar2(100) := '[!-]'; - l_right_side_regex varchar(400) := '([!-])([^!-].*)'; - l_right_side varchar2(4000); - - l_tags varchar2(4000) := a_tags; - l_result number :=1; - begin - --Validate that we have closed up all brackets - if regexp_count(l_tags,'\(') <> regexp_count(l_tags,'\)') then - l_result := 0; - end if; - - --Remove brackets as we dont evaluate expression only validate. - l_tags := replace(replace(l_tags,'('),')'); - - --Check if there are any left side operators for first in order from left to right - if regexp_count(l_tags,l_left_side_expression) > 0 then - --Extract left part of operator and remaining of string to right - l_left_side := regexp_replace(l_tags,l_left_side_regex,'\1'); - l_right_side := regexp_replace(l_tags,l_left_side_regex,'\2'); - - --If left side is null that means that we used left side operator without - -- left and right e.g. &test - if l_left_side is null then - l_result := 0; - else - --Extract right side from left side expression if there is any !- - --Remove first negation tag to see if there is double negation - l_left_side := regexp_replace(l_left_side,l_right_side_regex,'\2'); - end if; - - - --check that on right side there is no extra negation - if regexp_count(l_left_side,l_rigth_side_expression) > 0 then - l_result := 0; - end if; - - --Now process right side of string - if l_right_side is not null then - l_result := least(l_result,valid_tag_expression(l_right_side)); - else - l_result := 0; - end if; - else - --We just process single tag. - l_left_side := l_tags; - l_left_side := regexp_replace(l_left_side,l_right_side_regex,'\2'); - if regexp_count(l_left_side,l_rigth_side_expression) > 0 then - l_result := 0; - end if; - end if; - - return l_result; - end; + end ut_utils; / diff --git a/source/core/ut_utils.pks b/source/core/ut_utils.pks index 2975c3846..f7f7b4f00 100644 --- a/source/core/ut_utils.pks +++ b/source/core/ut_utils.pks @@ -477,7 +477,5 @@ create or replace package ut_utils authid definer is */ function interval_to_text(a_interval yminterval_unconstrained) return varchar2; - function valid_tag_expression(a_tags in varchar2) return number; - end ut_utils; / diff --git a/test/ut3_tester/core/test_ut_utils.pkb b/test/ut3_tester/core/test_ut_utils.pkb index 52c9cdff8..4ed718777 100644 --- a/test/ut3_tester/core/test_ut_utils.pkb +++ b/test/ut3_tester/core/test_ut_utils.pkb @@ -489,32 +489,5 @@ end; ut.expect(l_expected).to_equal(l_actual); end; - procedure valid_tag_expressions is - begin - ut.expect(1).to_equal(ut3_develop.ut_utils.valid_tag_expression('tag1')); - ut.expect(1).to_equal(ut3_develop.ut_utils.valid_tag_expression('tag1|tag2')); - ut.expect(1).to_equal(ut3_develop.ut_utils.valid_tag_expression('tag1&tag2')); - ut.expect(1).to_equal(ut3_develop.ut_utils.valid_tag_expression('!tag1')); - ut.expect(1).to_equal(ut3_develop.ut_utils.valid_tag_expression('tag1|!tag2')); - ut.expect(1).to_equal(ut3_develop.ut_utils.valid_tag_expression('tag1&!tag2')); - ut.expect(1).to_equal(ut3_develop.ut_utils.valid_tag_expression('!tag1|!tag2')); - ut.expect(1).to_equal(ut3_develop.ut_utils.valid_tag_expression('!tag1&!tag2')); - - ut.expect(1).to_equal(ut3_develop.ut_utils.valid_tag_expression('tag1,tag2')); - ut.expect(1).to_equal(ut3_develop.ut_utils.valid_tag_expression('-tag1')); - ut.expect(1).to_equal(ut3_develop.ut_utils.valid_tag_expression('tag1,-tag2')); - ut.expect(1).to_equal(ut3_develop.ut_utils.valid_tag_expression('-tag1,-tag2')); - - ut.expect(1).to_equal(ut3_develop.ut_utils.valid_tag_expression('(!tag1|!tag2)|tag3')); - ut.expect(1).to_equal(ut3_develop.ut_utils.valid_tag_expression('(!tag1&!tag2)|(tag3&tag4)')); - - ut.expect(0).to_equal(ut3_develop.ut_utils.valid_tag_expression('tag1|')); - ut.expect(0).to_equal(ut3_develop.ut_utils.valid_tag_expression('&!tag2')); - ut.expect(0).to_equal(ut3_develop.ut_utils.valid_tag_expression('!!tag1|!tag2')); - ut.expect(0).to_equal(ut3_develop.ut_utils.valid_tag_expression('!tag1&!tag2|')); - ut.expect(0).to_equal(ut3_develop.ut_utils.valid_tag_expression('((!tag1|!tag2)|tag3')); - - end; - end test_ut_utils; / diff --git a/test/ut3_tester/core/test_ut_utils.pks b/test/ut3_tester/core/test_ut_utils.pks index b87d3b2c6..4d83b5042 100644 --- a/test/ut3_tester/core/test_ut_utils.pks +++ b/test/ut3_tester/core/test_ut_utils.pks @@ -154,8 +154,6 @@ create or replace package test_ut_utils is --%test(returns text representation of interval year to month for custom interval) procedure int_conv_ym_date; - --%test(Test to validate different type of expressions passed as tags) - procedure valid_tag_expressions; --%endcontext From e87d39f992da9ff7fdb0cb2b653f6fc0380c6def Mon Sep 17 00:00:00 2001 From: Lukasz Wasylow Date: Wed, 29 Mar 2023 15:17:46 -0700 Subject: [PATCH 24/77] Adding validate function, with no calls --- source/core/ut_utils.pkb | 61 ++++++++++++++++++++++++++++++++++++++++ source/core/ut_utils.pks | 7 ++++- 2 files changed, 67 insertions(+), 1 deletion(-) diff --git a/source/core/ut_utils.pkb b/source/core/ut_utils.pkb index 162e50f8f..14d0bc148 100644 --- a/source/core/ut_utils.pkb +++ b/source/core/ut_utils.pkb @@ -989,6 +989,67 @@ create or replace package body ut_utils is return l_result; end; + function valid_tag_expression(a_tags in varchar2) return number is + t_left_side ut_varchar2_list := ut_varchar2_list('|','&',','); + t_right_side ut_varchar2_list := ut_varchar2_list('!','-'); + l_left_side_expression varchar2(100) := '[|&,]'; + l_left_side_regex varchar(400) := '([^|&,]*)[|&,](.*)'; + l_left_side varchar2(4000); + + l_rigth_side_expression varchar2(100) := '[!-]'; + l_right_side_regex varchar(400) := '([!-])([^!-].*)'; + l_right_side varchar2(4000); + + l_tags varchar2(4000) := a_tags; + l_result number :=1; + begin + --Validate that we have closed up all brackets + if regexp_count(l_tags,'\(') <> regexp_count(l_tags,'\)') then + l_result := 0; + end if; + + --Remove brackets as we dont evaluate expression only validate. + l_tags := replace(replace(l_tags,'('),')'); + + --Check if there are any left side operators for first in order from left to right + if regexp_count(l_tags,l_left_side_expression) > 0 then + --Extract left part of operator and remaining of string to right + l_left_side := regexp_replace(l_tags,l_left_side_regex,'\1'); + l_right_side := regexp_replace(l_tags,l_left_side_regex,'\2'); + + --If left side is null that means that we used left side operator without + -- left and right e.g. &test + if l_left_side is null then + l_result := 0; + else + --Extract right side from left side expression if there is any !- + --Remove first negation tag to see if there is double negation + l_left_side := regexp_replace(l_left_side,l_right_side_regex,'\2'); + end if; + + + --check that on right side there is no extra negation + if regexp_count(l_left_side,l_rigth_side_expression) > 0 then + l_result := 0; + end if; + + --Now process right side of string + if l_right_side is not null then + l_result := least(l_result,valid_tag_expression(l_right_side)); + else + l_result := 0; + end if; + else + --We just process single tag. + l_left_side := l_tags; + l_left_side := regexp_replace(l_left_side,l_right_side_regex,'\2'); + if regexp_count(l_left_side,l_rigth_side_expression) > 0 then + l_result := 0; + end if; + end if; + + return l_result; + end; end ut_utils; / diff --git a/source/core/ut_utils.pks b/source/core/ut_utils.pks index f7f7b4f00..06c6956b4 100644 --- a/source/core/ut_utils.pks +++ b/source/core/ut_utils.pks @@ -476,6 +476,11 @@ create or replace package ut_utils authid definer is * Return value of interval in plain english */ function interval_to_text(a_interval yminterval_unconstrained) return varchar2; - + + /* + * Return number 1 or 0 if the list of tags is valid expression + */ + function valid_tag_expression(a_tags in varchar2) return number; + end ut_utils; / From 97537de9629c19313ccc6157ac95f323a2de6656 Mon Sep 17 00:00:00 2001 From: Lukasz Wasylow Date: Wed, 29 Mar 2023 15:29:07 -0700 Subject: [PATCH 25/77] Remove a & from text --- source/api/ut_runner.pkb | 7 ++----- source/core/ut_utils.pkb | 2 +- test/ut3_tester/core/test_ut_utils.pkb | 27 ++++++++++++++++++++++++++ test/ut3_tester/core/test_ut_utils.pks | 3 +++ 4 files changed, 33 insertions(+), 6 deletions(-) diff --git a/source/api/ut_runner.pkb b/source/api/ut_runner.pkb index 94538eb69..a2a69f464 100644 --- a/source/api/ut_runner.pkb +++ b/source/api/ut_runner.pkb @@ -95,13 +95,10 @@ create or replace package body ut_runner is ut_event_manager.trigger_event(ut_event_manager.gc_initialize); ut_event_manager.trigger_event(ut_event_manager.gc_debug, ut_run_info()); - --TODO:Verify tag tag expression is valid - /* - if regexp_like(l_tags,'[&|]{2,}|[!-]{2,}|[!-][&|]|[^-&|!,]+[-!]|[-!|&][)]') - or (regexp_count(l_tags,'\(') <> regexp_count(l_tags,'\)')) then + if ut_utils.valid_tag_expression(l_tags) = 0 then raise_application_error(ut_utils.gc_invalid_tag_expression, 'Invalid Tag expression'); end if; - */ + if a_random_test_order_seed is not null then l_random_test_order_seed := a_random_test_order_seed; elsif a_random_test_order then diff --git a/source/core/ut_utils.pkb b/source/core/ut_utils.pkb index 14d0bc148..2a47d4d4b 100644 --- a/source/core/ut_utils.pkb +++ b/source/core/ut_utils.pkb @@ -1018,7 +1018,7 @@ create or replace package body ut_utils is l_right_side := regexp_replace(l_tags,l_left_side_regex,'\2'); --If left side is null that means that we used left side operator without - -- left and right e.g. &test + -- left and right e.g. |test if l_left_side is null then l_result := 0; else diff --git a/test/ut3_tester/core/test_ut_utils.pkb b/test/ut3_tester/core/test_ut_utils.pkb index 4ed718777..8a497b9f1 100644 --- a/test/ut3_tester/core/test_ut_utils.pkb +++ b/test/ut3_tester/core/test_ut_utils.pkb @@ -488,6 +488,33 @@ end; begin ut.expect(l_expected).to_equal(l_actual); end; + + procedure valid_tag_expressions is + begin + ut.expect(1).to_equal(ut3_develop.ut_utils.valid_tag_expression('tag1')); + ut.expect(1).to_equal(ut3_develop.ut_utils.valid_tag_expression('tag1|tag2')); + ut.expect(1).to_equal(ut3_develop.ut_utils.valid_tag_expression('tag1&tag2')); + ut.expect(1).to_equal(ut3_develop.ut_utils.valid_tag_expression('!tag1')); + ut.expect(1).to_equal(ut3_develop.ut_utils.valid_tag_expression('tag1|!tag2')); + ut.expect(1).to_equal(ut3_develop.ut_utils.valid_tag_expression('tag1&!tag2')); + ut.expect(1).to_equal(ut3_develop.ut_utils.valid_tag_expression('!tag1|!tag2')); + ut.expect(1).to_equal(ut3_develop.ut_utils.valid_tag_expression('!tag1&!tag2')); + + ut.expect(1).to_equal(ut3_develop.ut_utils.valid_tag_expression('tag1,tag2')); + ut.expect(1).to_equal(ut3_develop.ut_utils.valid_tag_expression('-tag1')); + ut.expect(1).to_equal(ut3_develop.ut_utils.valid_tag_expression('tag1,-tag2')); + ut.expect(1).to_equal(ut3_develop.ut_utils.valid_tag_expression('-tag1,-tag2')); + + ut.expect(1).to_equal(ut3_develop.ut_utils.valid_tag_expression('(!tag1|!tag2)|tag3')); + ut.expect(1).to_equal(ut3_develop.ut_utils.valid_tag_expression('(!tag1&!tag2)|(tag3&tag4)')); + + ut.expect(0).to_equal(ut3_develop.ut_utils.valid_tag_expression('tag1|')); + ut.expect(0).to_equal(ut3_develop.ut_utils.valid_tag_expression('&!tag2')); + ut.expect(0).to_equal(ut3_develop.ut_utils.valid_tag_expression('!!tag1|!tag2')); + ut.expect(0).to_equal(ut3_develop.ut_utils.valid_tag_expression('!tag1&!tag2|')); + ut.expect(0).to_equal(ut3_develop.ut_utils.valid_tag_expression('((!tag1|!tag2)|tag3')); + + end; end test_ut_utils; / diff --git a/test/ut3_tester/core/test_ut_utils.pks b/test/ut3_tester/core/test_ut_utils.pks index 4d83b5042..0f8545a45 100644 --- a/test/ut3_tester/core/test_ut_utils.pks +++ b/test/ut3_tester/core/test_ut_utils.pks @@ -157,5 +157,8 @@ create or replace package test_ut_utils is --%endcontext + --%test(Test to validate different type of expressions passed as tags) + procedure valid_tag_expressions; + end test_ut_utils; / From 69092f249427dccd39ff0df26848cbd873701ef9 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" Date: Thu, 30 Mar 2023 20:07:27 +0100 Subject: [PATCH 26/77] Updated project version after build [skip ci] --- docs/about/authors.md | 2 +- docs/about/license.md | 2 +- docs/about/project-details.md | 2 +- docs/about/support.md | 2 +- docs/index.md | 2 +- docs/userguide/advanced_data_comparison.md | 2 +- docs/userguide/annotations.md | 2 +- docs/userguide/best-practices.md | 2 +- docs/userguide/coverage.md | 2 +- docs/userguide/exception-reporting.md | 2 +- docs/userguide/expectations.md | 2 +- docs/userguide/getting-started.md | 2 +- docs/userguide/install.md | 2 +- docs/userguide/querying_suites.md | 2 +- docs/userguide/reporters.md | 2 +- docs/userguide/running-unit-tests.md | 2 +- docs/userguide/upgrade.md | 2 +- source/core/ut_utils.pks | 2 +- 18 files changed, 18 insertions(+), 18 deletions(-) diff --git a/docs/about/authors.md b/docs/about/authors.md index d132d2085..a47004a3c 100644 --- a/docs/about/authors.md +++ b/docs/about/authors.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4094--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4105--develop-blue.svg) ### utPLSQL v3 Major Contributors diff --git a/docs/about/license.md b/docs/about/license.md index 21b82ee41..8b6437299 100644 --- a/docs/about/license.md +++ b/docs/about/license.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4094--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4105--develop-blue.svg) # Version Information diff --git a/docs/about/project-details.md b/docs/about/project-details.md index 9b423d5a8..02eabab07 100644 --- a/docs/about/project-details.md +++ b/docs/about/project-details.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4094--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4105--develop-blue.svg) # utPLSQL Project Details diff --git a/docs/about/support.md b/docs/about/support.md index d534857fa..aacb289f2 100644 --- a/docs/about/support.md +++ b/docs/about/support.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4094--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4105--develop-blue.svg) # How to get support diff --git a/docs/index.md b/docs/index.md index d6884ba2d..5e30a9fc6 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4094--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4105--develop-blue.svg) ## What is utPLSQL diff --git a/docs/userguide/advanced_data_comparison.md b/docs/userguide/advanced_data_comparison.md index 2c31a9a89..8db920ee7 100644 --- a/docs/userguide/advanced_data_comparison.md +++ b/docs/userguide/advanced_data_comparison.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4094--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4105--develop-blue.svg) # Advanced data comparison diff --git a/docs/userguide/annotations.md b/docs/userguide/annotations.md index d193e2c20..31b2fd442 100644 --- a/docs/userguide/annotations.md +++ b/docs/userguide/annotations.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4094--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4105--develop-blue.svg) Annotations are used to configure tests and suites in a declarative way similar to modern OOP languages. This way, test configuration is stored along with the test logic inside the test package. No additional configuration files or tables are needed for test cases. The annotation names are based on popular testing frameworks such as JUnit. diff --git a/docs/userguide/best-practices.md b/docs/userguide/best-practices.md index 63c0e36e9..b83fb78c3 100644 --- a/docs/userguide/best-practices.md +++ b/docs/userguide/best-practices.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4094--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4105--develop-blue.svg) The following are best practices we at utPLSQL have learned about PL/SQL and Unit Testing. diff --git a/docs/userguide/coverage.md b/docs/userguide/coverage.md index f77717798..04b3db380 100644 --- a/docs/userguide/coverage.md +++ b/docs/userguide/coverage.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4094--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4105--develop-blue.svg) utPLSQL comes with a built-in coverage reporting engine. The code coverage reporting uses package DBMS_PROFILER (and DBMS_PLSQL_CODE_COVERAGE on Oracle database version 12.2 and above) provided with Oracle database. Code coverage is gathered for the following source types: diff --git a/docs/userguide/exception-reporting.md b/docs/userguide/exception-reporting.md index e237bf4ef..a7b0ae0a5 100644 --- a/docs/userguide/exception-reporting.md +++ b/docs/userguide/exception-reporting.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4094--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4105--develop-blue.svg) utPLSQL is responsible for handling exceptions wherever they occur in the test run. The framework is trapping most of the exceptions so that the test execution is not affected by individual tests or test packages throwing an exception. The framework provides a full stacktrace for every exception that was thrown. The reported stacktrace does not include any utPLSQL library calls in it. diff --git a/docs/userguide/expectations.md b/docs/userguide/expectations.md index 8fa525a69..2b436250e 100644 --- a/docs/userguide/expectations.md +++ b/docs/userguide/expectations.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4094--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4105--develop-blue.svg) ## Expectation concepts diff --git a/docs/userguide/getting-started.md b/docs/userguide/getting-started.md index d97d49464..37c22c882 100644 --- a/docs/userguide/getting-started.md +++ b/docs/userguide/getting-started.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4094--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4105--develop-blue.svg) # Getting started with TDD and utPLSQL diff --git a/docs/userguide/install.md b/docs/userguide/install.md index 934d07d72..00a74b288 100644 --- a/docs/userguide/install.md +++ b/docs/userguide/install.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4094--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4105--develop-blue.svg) ## Supported database versions diff --git a/docs/userguide/querying_suites.md b/docs/userguide/querying_suites.md index 9e16fd3e8..164c0cbc0 100644 --- a/docs/userguide/querying_suites.md +++ b/docs/userguide/querying_suites.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4094--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4105--develop-blue.svg) ## Obtaining information about suites diff --git a/docs/userguide/reporters.md b/docs/userguide/reporters.md index b3b0e338a..e92558e3d 100644 --- a/docs/userguide/reporters.md +++ b/docs/userguide/reporters.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4094--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4105--develop-blue.svg) utPLSQL provides several reporting formats. The sections below describe most of them. diff --git a/docs/userguide/running-unit-tests.md b/docs/userguide/running-unit-tests.md index e8e431980..978803b9a 100644 --- a/docs/userguide/running-unit-tests.md +++ b/docs/userguide/running-unit-tests.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4094--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4105--develop-blue.svg) utPLSQL framework provides two main entry points to run unit tests from within the database: diff --git a/docs/userguide/upgrade.md b/docs/userguide/upgrade.md index 47772937e..794e75070 100644 --- a/docs/userguide/upgrade.md +++ b/docs/userguide/upgrade.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4094--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4105--develop-blue.svg) # Upgrading from version 2 diff --git a/source/core/ut_utils.pks b/source/core/ut_utils.pks index f92fbc2b0..e79ce1fa6 100644 --- a/source/core/ut_utils.pks +++ b/source/core/ut_utils.pks @@ -21,7 +21,7 @@ create or replace package ut_utils authid definer is * */ - gc_version constant varchar2(50) := 'v3.1.14.4094-develop'; + gc_version constant varchar2(50) := 'v3.1.14.4105-develop'; subtype t_executable_type is varchar2(30); gc_before_all constant t_executable_type := 'beforeall'; From 2a0f99affc9b6b43dfa9e6f00c053c0eada27b14 Mon Sep 17 00:00:00 2001 From: Lukasz Wasylow Date: Fri, 31 Mar 2023 08:29:14 -0700 Subject: [PATCH 27/77] Extra changes and added tests --- source/core/ut_suite_cache_manager.pkb | 6 +- test/ut3_tester_helper/run_helper.pkb | 94 ++++++++++++++++++++++++++ test/ut3_user/api/test_ut_run.pkb | 33 +++++++++ test/ut3_user/api/test_ut_run.pks | 3 + 4 files changed, 133 insertions(+), 3 deletions(-) diff --git a/source/core/ut_suite_cache_manager.pkb b/source/core/ut_suite_cache_manager.pkb index ff18deabf..ca7945c74 100644 --- a/source/core/ut_suite_cache_manager.pkb +++ b/source/core/ut_suite_cache_manager.pkb @@ -273,9 +273,9 @@ create or replace package body ut_suite_cache_manager is '(\(|\)|\||\!|\&)?([^|&!-()]+)(\(|\)|\||\!|\&)?', q'[\1q'<\2>' member of tags\3]'); --replace operands to XPath - l_tags := REGEXP_REPLACE(l_tags, '\|',' or '); - l_tags := REGEXP_REPLACE(l_tags , '\&',' and '); - l_tags := REGEXP_REPLACE(l_tags,q'[(\!)(q'<[^|&!-]+>')( member of tags)]','\2 not \3'); + l_tags := REPLACE(l_tags, '|',' or '); + l_tags := REPLACE(l_tags , '&',' and '); + l_tags := REGEXP_REPLACE(l_tags,q'[(\!)(q'<[^|&!]+?>')( member of tags)]','\2 not \3'); l_tags := '('||l_tags||')'; return l_tags; end; diff --git a/test/ut3_tester_helper/run_helper.pkb b/test/ut3_tester_helper/run_helper.pkb index a132f4943..c73564798 100644 --- a/test/ut3_tester_helper/run_helper.pkb +++ b/test/ut3_tester_helper/run_helper.pkb @@ -343,9 +343,100 @@ create or replace package body run_helper is end; end test_package_3; ]'; + + execute immediate q'[create or replace package test_tag_pkg_1 is + --%suite + --%tags(suite1,release_3_1_13,development,complex,end_to_end) + --%suitepath(suite1) + --%rollback(manual) + + --%test(Test1 from test_tag_pkg_1) + --%tags(test1,development,fast) + procedure test1; + + --%test(Test2 from test_tag_pkg_1) + --%tags(test2,production,slow,patch_3_1_13) + procedure test2; + + end test_tag_pkg_1; + ]'; + + execute immediate q'[create or replace package body test_tag_pkg_1 is + procedure test1 is + begin + dbms_output.put_line('test_tag_pkg_1.test1 executed'); + end; + procedure test2 is + begin + dbms_output.put_line('test_tag_pkg_1.test2 executed'); + end; + end test_tag_pkg_1; + ]'; + + execute immediate q'[create or replace package test_tag_pkg_2 is + --%suite + --%tags(suite2,release_3_1_12,development,simple) + --%suitepath(suite1.suite2) + --%rollback(manual) + + --%test(Test3 from test_tag_pkg_2) + --%tags(test3,development,fast) + procedure test3; + + --%test(Test4 from test_tag_pkg_1) + --%tags(test4,production,slow) + procedure test4; + + end test_tag_pkg_2; + ]'; + + execute immediate q'[create or replace package body test_tag_pkg_2 is + procedure test3 is + begin + dbms_output.put_line('test_tag_pkg_2.test3 executed'); + end; + procedure test4 is + begin + dbms_output.put_line('test_tag_pkg_2.test4 executed'); + end; + end test_tag_pkg_2; + ]'; + + execute immediate q'[create or replace package test_tag_pkg_3 is + --%suite + --%tags(suite3,release_3_1_13,production,simple,end_to_end) + --%suitepath(suite3) + --%rollback(manual) + + --%test(Test5 from test_tag_pkg_3) + --%tags(test5,release_3_1_13,production,patch_3_1_13) + procedure test5; + + --%test(Test6 from test_tag_pkg_3) + --%tags(test6,development,patch_3_1_14) + procedure test6; + + end test_tag_pkg_3; + ]'; + + execute immediate q'[create or replace package body test_tag_pkg_3 is + procedure test5 is + begin + dbms_output.put_line('test_tag_pkg_3.test5 executed'); + end; + procedure test6 is + begin + dbms_output.put_line('test_tag_pkg_3.test6 executed'); + end; + end test_tag_pkg_3; + ]'; + execute immediate q'[grant execute on test_package_1 to public]'; execute immediate q'[grant execute on test_package_2 to public]'; execute immediate q'[grant execute on test_package_3 to public]'; + execute immediate q'[grant execute on test_tag_pkg_1 to public]'; + execute immediate q'[grant execute on test_tag_pkg_2 to public]'; + execute immediate q'[grant execute on test_tag_pkg_3 to public]'; end; procedure drop_ut3_user_tests is @@ -354,6 +445,9 @@ create or replace package body run_helper is execute immediate q'[drop package test_package_1]'; execute immediate q'[drop package test_package_2]'; execute immediate q'[drop package test_package_3]'; + execute immediate q'[drop package test_tag_pkg_1]'; + execute immediate q'[drop package test_tag_pkg_2]'; + execute immediate q'[drop package test_tag_pkg_3]'; end; procedure create_test_suite is diff --git a/test/ut3_user/api/test_ut_run.pkb b/test/ut3_user/api/test_ut_run.pkb index 6f6e229c8..ee420f310 100644 --- a/test/ut3_user/api/test_ut_run.pkb +++ b/test/ut3_user/api/test_ut_run.pkb @@ -1202,6 +1202,39 @@ procedure tag_exclude_run_fun_pth_lst_lg is ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_be_like( '%test_package_2.test2%executed%' ); end; + procedure tag_complex_expressions is + l_results ut3_develop.ut_varchar2_list; + begin + l_results := ut3_tester_helper.run_helper.run(a_tags => 'release_3_1_13&(fast|simple)'); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_be_like( '%test_tag_pkg_3.test5 executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_be_like( '%test_tag_pkg_3.test6 executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like( '%test_package_1.test1%executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like( '%test_package_1.test2%executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like( '%test_package_2.test3%executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like( '%test_package_2.test4%executed%' ); + + l_results := ut3_tester_helper.run_helper.run(a_tags => 'release_3_1_13&(!patch_3_1_13&!patch_3_1_14)'); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_be_like( '%test_tag_pkg_1.test1 executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like( '%test_package_1.test2%executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like( '%test_package_2.test3%executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like( '%test_package_2.test4%executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like( '%test_package_3.test5%executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like( '%test_package_3.test6%executed%' ); + + l_results := ut3_tester_helper.run_helper.run(a_tags => 'release_3_1_13&(patch_3_1_13&!slow)'); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_be_like( '%test_tag_pkg_3.test5 executed%' ); + + l_results := ut3_tester_helper.run_helper.run(a_tags => '(simple&end_to_end)|(development&fast)'); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_be_like( '%test_tag_pkg_1.test1 executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_be_like( '%test_tag_pkg_2.test3 executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_be_like( '%test_tag_pkg_3.test5 executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_be_like( '%test_tag_pkg_3.test6 executed%' ); + + l_results := ut3_tester_helper.run_helper.run(a_tags => '(!development&end_to_end)'); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_be_like( '%test_tag_pkg_3.test5 executed%' ); + + end; + procedure set_application_info is begin dbms_application_info.set_module( gc_module, gc_action ); diff --git a/test/ut3_user/api/test_ut_run.pks b/test/ut3_user/api/test_ut_run.pks index db02ee8f8..40e870fe4 100644 --- a/test/ut3_user/api/test_ut_run.pks +++ b/test/ut3_user/api/test_ut_run.pks @@ -259,6 +259,9 @@ create or replace package test_ut_run is --%test(Runs tests from given tags and exclude tags - Legacy Notation) procedure tag_include_exclude_run_fun_lg; + --%test(Runs tests suing complex expressions) + procedure tag_complex_expressions; + --%endcontext --%context(ut3_info context) From b30688cae39ea3917c92427528a5649022e76a4b Mon Sep 17 00:00:00 2001 From: Lukasz Wasylow Date: Sat, 1 Apr 2023 08:31:13 -0700 Subject: [PATCH 28/77] Address sonar coverage issues. --- source/core/ut_suite_cache_manager.pkb | 14 ++++++++------ source/core/ut_utils.pkb | 14 ++++++++++---- 2 files changed, 18 insertions(+), 10 deletions(-) diff --git a/source/core/ut_suite_cache_manager.pkb b/source/core/ut_suite_cache_manager.pkb index ca7945c74..2ccf20721 100644 --- a/source/core/ut_suite_cache_manager.pkb +++ b/source/core/ut_suite_cache_manager.pkb @@ -227,11 +227,10 @@ create or replace package body ut_suite_cache_manager is - = NOT we will perform a replace of that characters into new notation. - || = OR - && = AND - ^ = NOT + | = OR + & = AND + ! = NOT */ - --TODO: How do we prevent when old notation reach 4k an new will be longer? function replace_legacy_tag_notation(a_tags varchar2 ) return varchar2 is l_tags ut_varchar2_list := ut_utils.string_to_table(a_tags,','); @@ -331,8 +330,11 @@ with ) ) t where c.id = t.id and r_num = 1 ]'; - execute immediate l_sql bulk collect into l_suite_tags using a_suite_items; - return l_suite_tags; + execute immediate l_sql bulk collect into l_suite_tags using a_suite_items; + return l_suite_tags; + exception when others then + --If the dynamic SQL fails we will fall gracefully with meaningfull message + raise_application_error(ut_utils.gc_invalid_tag_expression, 'Tag expression, causing error. If expression is correct please report error.'); end; /* diff --git a/source/core/ut_utils.pkb b/source/core/ut_utils.pkb index 751971202..3747b029a 100644 --- a/source/core/ut_utils.pkb +++ b/source/core/ut_utils.pkb @@ -990,15 +990,21 @@ create or replace package body ut_utils is return l_result; end; + /* + Purpose of this function is to break down the tag expressions + We can separate operators on left and rigth side. + Left ones are AND and OR as they require an operator on left side to + be valid. Right side is NOT. + In each iteration we breakdown string into parts + + */ function valid_tag_expression(a_tags in varchar2) return number is - t_left_side ut_varchar2_list := ut_varchar2_list('|','&',','); - t_right_side ut_varchar2_list := ut_varchar2_list('!','-'); l_left_side_expression varchar2(100) := '[|&,]'; - l_left_side_regex varchar(400) := '([^|&,]*)[|&,](.*)'; + l_left_side_regex varchar2(400) := '([^|&,]*)[|&,](.*)'; l_left_side varchar2(4000); l_rigth_side_expression varchar2(100) := '[!-]'; - l_right_side_regex varchar(400) := '([!-])([^!-].*)'; + l_right_side_regex varchar2(400) := '([!-])([^!-].*)'; l_right_side varchar2(4000); l_tags varchar2(4000) := a_tags; From 0c41a0f22816b901e0b2adca5441e462feedb88a Mon Sep 17 00:00:00 2001 From: Lukasz Wasylow Date: Sat, 1 Apr 2023 09:19:43 -0700 Subject: [PATCH 29/77] Adding tests covering exception of invalid tags --- source/core/ut_suite_cache_manager.pkb | 5 +---- test/ut3_user/api/test_ut_run.pkb | 30 +++++++++++++++++++++++++- test/ut3_user/api/test_ut_run.pks | 4 ++++ 3 files changed, 34 insertions(+), 5 deletions(-) diff --git a/source/core/ut_suite_cache_manager.pkb b/source/core/ut_suite_cache_manager.pkb index 2ccf20721..152b5c0f9 100644 --- a/source/core/ut_suite_cache_manager.pkb +++ b/source/core/ut_suite_cache_manager.pkb @@ -331,10 +331,7 @@ with ) t where c.id = t.id and r_num = 1 ]'; execute immediate l_sql bulk collect into l_suite_tags using a_suite_items; - return l_suite_tags; - exception when others then - --If the dynamic SQL fails we will fall gracefully with meaningfull message - raise_application_error(ut_utils.gc_invalid_tag_expression, 'Tag expression, causing error. If expression is correct please report error.'); + return l_suite_tags; end; /* diff --git a/test/ut3_user/api/test_ut_run.pkb b/test/ut3_user/api/test_ut_run.pkb index ee420f310..84dd11f9c 100644 --- a/test/ut3_user/api/test_ut_run.pkb +++ b/test/ut3_user/api/test_ut_run.pkb @@ -1223,16 +1223,44 @@ procedure tag_exclude_run_fun_pth_lst_lg is l_results := ut3_tester_helper.run_helper.run(a_tags => 'release_3_1_13&(patch_3_1_13&!slow)'); ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_be_like( '%test_tag_pkg_3.test5 executed%' ); - + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like( '%test_package_1.test1%executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like( '%test_package_1.test2%executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like( '%test_package_2.test3%executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like( '%test_package_2.test4%executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like( '%test_package_3.test6%executed%' ); + l_results := ut3_tester_helper.run_helper.run(a_tags => '(simple&end_to_end)|(development&fast)'); ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_be_like( '%test_tag_pkg_1.test1 executed%' ); ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_be_like( '%test_tag_pkg_2.test3 executed%' ); ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_be_like( '%test_tag_pkg_3.test5 executed%' ); ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_be_like( '%test_tag_pkg_3.test6 executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like( '%test_package_1.test2%executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like( '%test_package_2.test4%executed%' ); l_results := ut3_tester_helper.run_helper.run(a_tags => '(!development&end_to_end)'); ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_be_like( '%test_tag_pkg_3.test5 executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like( '%test_package_1.test1%executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like( '%test_package_1.test2%executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like( '%test_package_2.test3%executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like( '%test_package_2.test4%executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like( '%test_package_3.test6%executed%' ); + + end; + + procedure invalid_tag_expression is + l_results ut3_develop.ut_varchar2_list; + begin + l_results := ut3_tester_helper.run_helper.run(a_tags => '(!!development&end_to_end)'); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_match('^\s*invalid_tag_expression \[[,\.0-9]+ sec\]\s*$','m'); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like('%(FAILED -%'); + + l_results := ut3_tester_helper.run_helper.run(a_tags => '(!development&&end_to_end)'); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_match('^\s*invalid_tag_expression \[[,\.0-9]+ sec\]\s*$','m'); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like('%(FAILED -%'); + l_results := ut3_tester_helper.run_helper.run(a_tags => '(!development&end_to_end|)'); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_match('^\s*invalid_tag_expression \[[,\.0-9]+ sec\]\s*$','m'); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like('%(FAILED -%'); end; procedure set_application_info is diff --git a/test/ut3_user/api/test_ut_run.pks b/test/ut3_user/api/test_ut_run.pks index 40e870fe4..c57788bff 100644 --- a/test/ut3_user/api/test_ut_run.pks +++ b/test/ut3_user/api/test_ut_run.pks @@ -262,6 +262,10 @@ create or replace package test_ut_run is --%test(Runs tests suing complex expressions) procedure tag_complex_expressions; + --%test(Testing invalid tag expression) + --%throws(-20219) + procedure invalid_tag_expression; + --%endcontext --%context(ut3_info context) From 543685dc94676110099af0b8af1a155918725cd5 Mon Sep 17 00:00:00 2001 From: Lukasz Wasylow Date: Sat, 1 Apr 2023 11:26:17 -0700 Subject: [PATCH 30/77] Removing that , we will not implement that, there is no benefit at the moment. --- docs/userguide/annotations.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/docs/userguide/annotations.md b/docs/userguide/annotations.md index f3cf73d4d..4017521b5 100644 --- a/docs/userguide/annotations.md +++ b/docs/userguide/annotations.md @@ -1662,8 +1662,6 @@ end ut_sample_test; Tag expressions are boolean expressions with the operators !, & and |. In addition, ( and ) can be used to adjust for operator precedence. -Two special expressions are supported, any() and none(), which select all tests with any tags at all, and all tests without any tags, respectively. These special expressions may be combined with other expressions just like normal tags. - | Operator | Meaning | | -------- | --------| | ! | not | From 0d3cfa166beccb14b19debffa99bb392f85d8bbb Mon Sep 17 00:00:00 2001 From: Lukasz Wasylow Date: Sat, 1 Apr 2023 11:30:00 -0700 Subject: [PATCH 31/77] Removing force --- source/core/types/ut_run.tps | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/core/types/ut_run.tps b/source/core/types/ut_run.tps index 8ce80d2a1..1878a2d46 100644 --- a/source/core/types/ut_run.tps +++ b/source/core/types/ut_run.tps @@ -1,4 +1,4 @@ -create or replace type ut_run force under ut_suite_item ( +create or replace type ut_run under ut_suite_item ( /* utPLSQL - Version 3 Copyright 2016 - 2021 utPLSQL Project From 20e317742e00f5299b2abd2675160d4b2cc38b86 Mon Sep 17 00:00:00 2001 From: Lukasz Wasylow Date: Sun, 9 Apr 2023 18:47:21 -0700 Subject: [PATCH 32/77] Changing to use Dijkstra algorithm to parse infix notation into postfix ( Reverse Polish Notation). This allows us to more flexibility of using boolean expressions and not limited to flaky regex. --- source/api/ut_runner.pkb | 4 - source/core/types/ut_stack.tpb | 58 ++++++ source/core/types/ut_stack.tps | 26 +++ source/core/ut_suite_cache_manager.pkb | 21 +-- source/core/ut_utils.pkb | 245 ++++++++++++++++++++++++- source/core/ut_utils.pks | 25 +++ source/install.sql | 2 + test/ut3_user/api/test_ut_run.pkb | 2 +- 8 files changed, 363 insertions(+), 20 deletions(-) create mode 100644 source/core/types/ut_stack.tpb create mode 100644 source/core/types/ut_stack.tps diff --git a/source/api/ut_runner.pkb b/source/api/ut_runner.pkb index a2a69f464..3d4550cf9 100644 --- a/source/api/ut_runner.pkb +++ b/source/api/ut_runner.pkb @@ -95,10 +95,6 @@ create or replace package body ut_runner is ut_event_manager.trigger_event(ut_event_manager.gc_initialize); ut_event_manager.trigger_event(ut_event_manager.gc_debug, ut_run_info()); - if ut_utils.valid_tag_expression(l_tags) = 0 then - raise_application_error(ut_utils.gc_invalid_tag_expression, 'Invalid Tag expression'); - end if; - if a_random_test_order_seed is not null then l_random_test_order_seed := a_random_test_order_seed; elsif a_random_test_order then diff --git a/source/core/types/ut_stack.tpb b/source/core/types/ut_stack.tpb new file mode 100644 index 000000000..a7f7ab0c2 --- /dev/null +++ b/source/core/types/ut_stack.tpb @@ -0,0 +1,58 @@ +create or replace type body ut_stack as + /* + utPLSQL - Version 3 + Copyright 2016 - 2021 utPLSQL Project + + Licensed under the Apache License, Version 2.0 (the "License"): + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + */ + + constructor function ut_stack( self in out nocopy ut_stack) return self as result is + begin + self.tokens := ut_varchar2_list(); + self.top := 0; + return; + end ut_stack; + + member function peek(self in out nocopy ut_stack) return varchar2 is + l_token varchar2(32767); + begin + if self.tokens.count =0 or self.tokens is null then + l_token := null; + else + l_token := self.tokens(self.tokens.last); + end if; + return l_token; + end; + + member procedure push(self in out nocopy ut_stack, a_token varchar2) is + begin + self.tokens.extend; + self.tokens(self.tokens.last) := a_token; + self.top := self.tokens.count; + end push; + + member procedure pop(self in out nocopy ut_stack,a_cnt in integer default 1) is + begin + self.tokens.trim(a_cnt); + self.top := self.tokens.count; + end pop; + + member function pop(self in out nocopy ut_stack) return varchar2 is + l_token varchar2(32767) := self.tokens(self.tokens.last); + begin + self.pop(); + return l_token; + end; +end; +/ + diff --git a/source/core/types/ut_stack.tps b/source/core/types/ut_stack.tps new file mode 100644 index 000000000..9851b9cc5 --- /dev/null +++ b/source/core/types/ut_stack.tps @@ -0,0 +1,26 @@ +create or replace type ut_stack as object ( + top integer, + tokens ut_varchar2_list, + /* + utPLSQL - Version 3 + Copyright 2016 - 2021 utPLSQL Project + + Licensed under the Apache License, Version 2.0 (the "License"): + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + */ + constructor function ut_stack( self in out nocopy ut_stack) return self as result, + member function peek(self in out nocopy ut_stack) return varchar2, + member procedure push(self in out nocopy ut_stack, a_token varchar2), + member procedure pop(self in out nocopy ut_stack,a_cnt in integer default 1), + member function pop(self in out nocopy ut_stack) return varchar2 +) + diff --git a/source/core/ut_suite_cache_manager.pkb b/source/core/ut_suite_cache_manager.pkb index 152b5c0f9..ae5460ef6 100644 --- a/source/core/ut_suite_cache_manager.pkb +++ b/source/core/ut_suite_cache_manager.pkb @@ -234,11 +234,11 @@ create or replace package body ut_suite_cache_manager is function replace_legacy_tag_notation(a_tags varchar2 ) return varchar2 is l_tags ut_varchar2_list := ut_utils.string_to_table(a_tags,','); - l_tags_include varchar2(2000); - l_tags_exclude varchar2(2000); + l_tags_include varchar2(4000); + l_tags_exclude varchar2(4000); l_return_tag varchar2(4000); begin - select listagg( t.column_value,' | ') + select listagg( t.column_value,'|') within group( order by column_value) into l_tags_include from table(l_tags) t @@ -268,15 +268,14 @@ create or replace package body ut_suite_cache_manager is if instr(l_tags,',') > 0 or instr(l_tags,'-') > 0 then l_tags := replace(replace_legacy_tag_notation(l_tags),' '); end if; - l_tags := REGEXP_REPLACE(l_tags, - '(\(|\)|\||\!|\&)?([^|&!-()]+)(\(|\)|\||\!|\&)?', - q'[\1q'<\2>' member of tags\3]'); - --replace operands to XPath + l_tags := ut_utils.convert_postfix_to_infix_where_sql(ut_utils.shunt_logical_expression(l_tags)); l_tags := REPLACE(l_tags, '|',' or '); - l_tags := REPLACE(l_tags , '&',' and '); - l_tags := REGEXP_REPLACE(l_tags,q'[(\!)(q'<[^|&!]+?>')( member of tags)]','\2 not \3'); - l_tags := '('||l_tags||')'; - return l_tags; + l_tags := REPLACE(l_tags ,'&',' and '); + l_tags := REPLACE(l_tags ,'!','not'); + return l_tags; + exception + when ut_utils.ex_invalid_tag_expression then + raise_application_error(ut_utils.gc_invalid_tag_expression, 'Invalid Tag expression'); end; /* diff --git a/source/core/ut_utils.pkb b/source/core/ut_utils.pkb index 3747b029a..c09c15221 100644 --- a/source/core/ut_utils.pkb +++ b/source/core/ut_utils.pkb @@ -24,6 +24,16 @@ create or replace package body ut_utils is gc_full_valid_xml_name constant varchar2(50) := '^([_a-zA-Z])([_a-zA-Z0-9\.-])*$'; gc_owner_hash constant integer(11) := dbms_utility.get_hash_value( ut_owner(), 0, power(2,31)-1); + /** + * Constants use in postfix and infix transformations + */ + gc_operators constant ut_varchar2_list := ut_varchar2_list('|','&','!'); + gc_unary_operator constant ut_varchar2_list := ut_varchar2_list('!'); -- right side associative operator + gc_binary_operator constant ut_varchar2_list := ut_varchar2_list('|','&'); -- left side associative operator + + type t_precedence_table is table of number index by varchar2(1); + g_precedence t_precedence_table; + function surround_with(a_value varchar2, a_quote_char varchar2) return varchar2 is begin return case when a_quote_char is not null then a_quote_char||a_value||a_quote_char else a_value end; @@ -999,12 +1009,12 @@ create or replace package body ut_utils is */ function valid_tag_expression(a_tags in varchar2) return number is - l_left_side_expression varchar2(100) := '[|&,]'; - l_left_side_regex varchar2(400) := '([^|&,]*)[|&,](.*)'; + l_left_side_expression varchar2(10) := '[|&,]'; + l_left_side_regex varchar2(50) := '([^|&,]*)[|&,](.*)'; l_left_side varchar2(4000); - l_rigth_side_expression varchar2(100) := '[!-]'; - l_right_side_regex varchar2(400) := '([!-])([^!-].*)'; + l_rigth_side_expression varchar2(10) := '[!-]'; + l_right_side_regex varchar2(50) := '([!-])([^!-].*)'; l_right_side varchar2(4000); l_tags varchar2(4000) := a_tags; @@ -1058,5 +1068,232 @@ create or replace package body ut_utils is return l_result; end; + procedure build_tag_expression_filter(a_tags in varchar2,a_expression_tab in out t_expression_tab,a_parent_id varchar2 default null) is + l_left_side_expression varchar2(10) := '[|&,]'; + l_left_side_regex varchar2(50) := '([^|&,]*)([|&,])(.*)'; + l_left_side varchar2(4000); + + l_rigth_side_expression varchar2(10) := '[!-]'; + l_right_side_regex varchar2(50) := '([!-])([^!-].*)'; + l_right_side varchar2(4000); + + l_tags varchar2(4000) := a_tags; + l_result number :=1; + l_expression_rec t_expression_rec; + + begin + if a_expression_tab is null then + a_expression_tab := t_expression_tab(); + end if; + + l_expression_rec.id := sys_guid(); + l_expression_rec.parent_id := a_parent_id; + + if instr(substr(l_tags,1,1),'(',1,1) + instr(substr(l_tags,-1,1),')',-1,1) = 2 then + + if regexp_count(l_tags,l_right_side_regex) = 1 then + l_expression_rec.negated :=1; + l_tags := trim (leading '!' from l_tags); + end if; + + l_expression_rec.left_bracket := 1; + l_tags := trim(leading '(' from l_tags); + l_expression_rec.right_bracket := 1; + l_tags := trim(trailing ')' from l_tags); + end if; + + + --Check if there are any left side operators for first in order from left to right + if regexp_count(l_tags,l_left_side_expression) > 0 then + --Extract left part of operator and remaining of string to right + + --if there are bracketc extract it and record it + + l_left_side := regexp_replace(l_tags,l_left_side_regex,'\1'); + l_expression_rec.log_operator := regexp_replace(l_tags,l_left_side_regex,'\2'); + l_right_side := regexp_replace(l_tags,l_left_side_regex,'\3'); + a_expression_tab.extend; + a_expression_tab(a_expression_tab.last) := l_expression_rec; + + build_tag_expression_filter(l_left_side,a_expression_tab,l_expression_rec.id); + build_tag_expression_filter(l_right_side,a_expression_tab,l_expression_rec.id); + + else + if instr(substr(l_tags,1,1),'(',1,1) + instr(substr(l_tags,-1,1),')',-1,1) = 2 then + + if regexp_count(l_tags,l_right_side_regex) = 1 then + l_expression_rec.negated :=1; + l_tags := trim (leading '!' from l_tags); + end if; + + l_expression_rec.left_bracket := 1; + l_tags := trim(leading '(' from l_tags); + l_expression_rec.right_bracket := 1; + l_tags := trim(trailing ')' from l_tags); + end if; + l_expression_rec.expression := l_tags; + a_expression_tab.extend; + a_expression_tab(a_expression_tab.last) := l_expression_rec; + end if; + + end; + + /* + https://stackoverflow.com/questions/29634992/shunting-yard-validate-expression + */ + function shunt_logical_expression(a_tags in varchar2) return ut_varchar2_list is + l_tags varchar2(32767) := a_tags; + l_operator_stack ut_stack := ut_stack(); + l_input_tokens ut_varchar2_list := ut_varchar2_list(); + l_rnp_tokens ut_varchar2_list := ut_varchar2_list(); + l_token varchar2(32767); + l_expect_operand boolean := true; + l_expect_operator boolean := false; + begin + --Tokenize a string into operators and tags + select regexp_substr(l_tags,'([^!()|&]+)|([!()|&])', 1, level) as string_parts + bulk collect into l_input_tokens + from dual connect by regexp_substr (l_tags, '([^!()|&]+)|([!()|&])', 1, level) is not null; + + --Exuecute modified shunting algorithm + for token in 1..l_input_tokens.count loop + l_token := l_input_tokens(token); + if (l_token member of gc_operators and l_token member of gc_binary_operator) then + if not(l_expect_operator) then + raise ex_invalid_tag_expression; + end if; + while l_operator_stack.top > 0 and (g_precedence(l_operator_stack.peek) > g_precedence(l_token)) loop + l_rnp_tokens.extend; + l_rnp_tokens(l_rnp_tokens.last) := l_operator_stack.pop; + end loop; + l_operator_stack.push(l_input_tokens(token)); + l_expect_operand := true; + l_expect_operator:= false; + elsif (l_token member of gc_operators and l_token member of gc_unary_operator) then + if not(l_expect_operand) then + raise ex_invalid_tag_expression; + end if; + l_operator_stack.push(l_input_tokens(token)); + l_expect_operand := true; + l_expect_operator:= false; + elsif l_token = '(' then + if not(l_expect_operand) then + raise ex_invalid_tag_expression; + end if; + l_operator_stack.push(l_input_tokens(token)); + l_expect_operand := true; + l_expect_operator:= false; + elsif l_token = ')' then + if not(l_expect_operator) then + raise ex_invalid_tag_expression; + end if; + while l_operator_stack.peek <> '(' loop + l_rnp_tokens.extend; + l_rnp_tokens(l_rnp_tokens.last) := l_operator_stack.pop; + end loop; + l_operator_stack.pop; --Pop the open bracket and discard it + l_expect_operand := false; + l_expect_operator:= true; + else + if not(l_expect_operand) then + raise ex_invalid_tag_expression; + end if; + l_rnp_tokens.extend; + l_rnp_tokens(l_rnp_tokens.last) :=l_token; + l_expect_operator := true; + l_expect_operand := false; + end if; + + end loop; + + while l_operator_stack.top > 0 loop + if l_operator_stack.peek in ('(',')') then + raise ex_invalid_tag_expression; + end if; + l_rnp_tokens.extend; + l_rnp_tokens(l_rnp_tokens.last):=l_operator_stack.pop; + end loop; + + return l_rnp_tokens; + end shunt_logical_expression; + + procedure shunt_logical_expression(a_tags in varchar2) is + a_postfix ut_varchar2_list; + begin + a_postfix := ut_utils.shunt_logical_expression(a_tags); + end shunt_logical_expression; + + function convert_postfix_to_infix(a_postfix_exp in ut_varchar2_list) + return varchar2 is + l_infix_stack ut_stack := ut_stack(); + l_right_side varchar2(32767); + l_left_side varchar2(32767); + l_infix_exp varchar2(32767); + begin + for i in 1..a_postfix_exp.count loop + --If token is operand but also single tag + if a_postfix_exp(i) not member of gc_operators then --its operand + l_infix_stack.push(a_postfix_exp(i)); + --If token is unary operator not + elsif a_postfix_exp(i) member of gc_unary_operator then + l_right_side := l_infix_stack.pop; + l_infix_exp := '('||a_postfix_exp(i)||l_right_side||')'; + l_infix_stack.push(l_infix_exp); + --If token is binary operator + elsif a_postfix_exp(i) member of gc_binary_operator then + l_right_side := l_infix_stack.pop; + l_left_side := l_infix_stack.pop; + l_infix_exp := '('||l_left_side||a_postfix_exp(i)||l_right_side||')'; + l_infix_stack.push(l_infix_exp); + else + null; + end if; + end loop; + + return l_infix_stack.pop; + end; + + function convert_postfix_to_infix_where_sql(a_postfix_exp in ut_varchar2_list) + return varchar2 is + l_infix_stack ut_stack := ut_stack(); + l_right_side varchar2(32767); + l_left_side varchar2(32767); + l_infix_exp varchar2(32767); + l_member_token varchar2(20) := ' member of tags'; + begin + for i in 1..a_postfix_exp.count loop + --If token is operand but also single tag + if regexp_count(a_postfix_exp(i),'[!()|&]') = 0 then + l_infix_stack.push(q'[']'||a_postfix_exp(i)||q'[']'||l_member_token); + --If token is operand but containing other expressions + elsif a_postfix_exp(i) not member of gc_operators then + l_infix_stack.push(a_postfix_exp(i)); + --If token is unary operator not + elsif a_postfix_exp(i) member of gc_unary_operator then + l_right_side := l_infix_stack.pop; + l_infix_exp := a_postfix_exp(i)||'('||l_right_side||')'; + l_infix_stack.push(l_infix_exp); + --If token is binary operator + elsif a_postfix_exp(i) member of gc_binary_operator then + l_right_side := l_infix_stack.pop; + l_left_side := l_infix_stack.pop; + l_infix_exp := '('||l_left_side||a_postfix_exp(i)||l_right_side||')'; + l_infix_stack.push(l_infix_exp); + else + null; + end if; + end loop; + + return l_infix_stack.pop; + end; + +begin + --Define operator precedence + g_precedence('!'):=4; + g_precedence('&'):=3; + g_precedence('|'):=2; + g_precedence(')'):=1; + g_precedence('('):=1; + end ut_utils; / diff --git a/source/core/ut_utils.pks b/source/core/ut_utils.pks index e84818cba..2212454f4 100644 --- a/source/core/ut_utils.pks +++ b/source/core/ut_utils.pks @@ -482,5 +482,30 @@ create or replace package ut_utils authid definer is */ function valid_tag_expression(a_tags in varchar2) return number; + /* + * Return number 1 or 0 if the list of tags is valid expression + */ + procedure build_tag_expression_filter(a_tags in varchar2,a_expression_tab in out t_expression_tab,a_parent_id varchar2 default null); + + /* + * Function that uses Dijkstra algorithm to parse mathematical and logical expression + * and return a list of elements in Reverse Polish Notation ( postfix ) + * As part of execution it will validate expression. + */ + function shunt_logical_expression(a_tags in varchar2) return ut_varchar2_list; + + procedure shunt_logical_expression(a_tags in varchar2); + + /* + * Function that converts postfix notation into infix + */ + function convert_postfix_to_infix(a_postfix_exp in ut_varchar2_list) return varchar2; + + /* + * Function that converts postfix notation into infix and creating a string of sql filter + * that checking a tags collections for tags according to posted logic. + */ + function convert_postfix_to_infix_where_sql(a_postfix_exp in ut_varchar2_list) return varchar2; + end ut_utils; / diff --git a/source/install.sql b/source/install.sql index 0873d6a1e..a54977f94 100644 --- a/source/install.sql +++ b/source/install.sql @@ -50,6 +50,8 @@ create or replace context &&ut3_owner._info using &&ut3_owner..ut_session_contex @@install_component.sql 'core/types/ut_key_value_pairs.tps' @@install_component.sql 'core/types/ut_reporter_info.tps' @@install_component.sql 'core/types/ut_reporters_info.tps' +@@install_component.sql 'core/types/ut_stack.tps' +@@install_component.sql 'core/types/ut_stack.tpb' @@install_component.sql 'core/ut_utils.pks' @@install_component.sql 'core/ut_metadata.pks' @@install_component.sql 'core/ut_savepoint_seq.sql' diff --git a/test/ut3_user/api/test_ut_run.pkb b/test/ut3_user/api/test_ut_run.pkb index 84dd11f9c..7eb377f4b 100644 --- a/test/ut3_user/api/test_ut_run.pkb +++ b/test/ut3_user/api/test_ut_run.pkb @@ -1250,7 +1250,7 @@ procedure tag_exclude_run_fun_pth_lst_lg is procedure invalid_tag_expression is l_results ut3_develop.ut_varchar2_list; begin - l_results := ut3_tester_helper.run_helper.run(a_tags => '(!!development&end_to_end)'); + l_results := ut3_tester_helper.run_helper.run(a_tags => '(!development!&end_to_end)'); ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_match('^\s*invalid_tag_expression \[[,\.0-9]+ sec\]\s*$','m'); ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like('%(FAILED -%'); From f51cc993be376e456758b73a21680591fa9130d9 Mon Sep 17 00:00:00 2001 From: Lukasz Wasylow Date: Sun, 9 Apr 2023 18:52:21 -0700 Subject: [PATCH 33/77] Missing slash at end of type --- source/core/types/ut_stack.tps | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/core/types/ut_stack.tps b/source/core/types/ut_stack.tps index 9851b9cc5..7b8c145c2 100644 --- a/source/core/types/ut_stack.tps +++ b/source/core/types/ut_stack.tps @@ -23,4 +23,4 @@ create or replace type ut_stack as object ( member procedure pop(self in out nocopy ut_stack,a_cnt in integer default 1), member function pop(self in out nocopy ut_stack) return varchar2 ) - +/ \ No newline at end of file From 4b8e2ab124d7819b2a8a442b9fcd31d6d5d5666e Mon Sep 17 00:00:00 2001 From: Lukasz Wasylow Date: Sun, 9 Apr 2023 21:26:13 -0700 Subject: [PATCH 34/77] Cleanup. Fix object name length. --- source/core/ut_suite_cache_manager.pkb | 2 +- source/core/ut_utils.pkb | 144 +------------------------ source/core/ut_utils.pks | 12 +-- 3 files changed, 5 insertions(+), 153 deletions(-) diff --git a/source/core/ut_suite_cache_manager.pkb b/source/core/ut_suite_cache_manager.pkb index ae5460ef6..aea8c667e 100644 --- a/source/core/ut_suite_cache_manager.pkb +++ b/source/core/ut_suite_cache_manager.pkb @@ -268,7 +268,7 @@ create or replace package body ut_suite_cache_manager is if instr(l_tags,',') > 0 or instr(l_tags,'-') > 0 then l_tags := replace(replace_legacy_tag_notation(l_tags),' '); end if; - l_tags := ut_utils.convert_postfix_to_infix_where_sql(ut_utils.shunt_logical_expression(l_tags)); + l_tags := ut_utils.convert_postfix_to_infix(ut_utils.shunt_logical_expression(l_tags)); l_tags := REPLACE(l_tags, '|',' or '); l_tags := REPLACE(l_tags ,'&',' and '); l_tags := REPLACE(l_tags ,'!','not'); diff --git a/source/core/ut_utils.pkb b/source/core/ut_utils.pkb index c09c15221..028a0c756 100644 --- a/source/core/ut_utils.pkb +++ b/source/core/ut_utils.pkb @@ -1000,144 +1000,6 @@ create or replace package body ut_utils is return l_result; end; - /* - Purpose of this function is to break down the tag expressions - We can separate operators on left and rigth side. - Left ones are AND and OR as they require an operator on left side to - be valid. Right side is NOT. - In each iteration we breakdown string into parts - - */ - function valid_tag_expression(a_tags in varchar2) return number is - l_left_side_expression varchar2(10) := '[|&,]'; - l_left_side_regex varchar2(50) := '([^|&,]*)[|&,](.*)'; - l_left_side varchar2(4000); - - l_rigth_side_expression varchar2(10) := '[!-]'; - l_right_side_regex varchar2(50) := '([!-])([^!-].*)'; - l_right_side varchar2(4000); - - l_tags varchar2(4000) := a_tags; - l_result number :=1; - begin - --Validate that we have closed up all brackets - if regexp_count(l_tags,'\(') <> regexp_count(l_tags,'\)') then - l_result := 0; - end if; - - --Remove brackets as we dont evaluate expression only validate. - l_tags := replace(replace(l_tags,'('),')'); - - --Check if there are any left side operators for first in order from left to right - if regexp_count(l_tags,l_left_side_expression) > 0 then - --Extract left part of operator and remaining of string to right - l_left_side := regexp_replace(l_tags,l_left_side_regex,'\1'); - l_right_side := regexp_replace(l_tags,l_left_side_regex,'\2'); - - --If left side is null that means that we used left side operator without - -- left and right e.g. |test - if l_left_side is null then - l_result := 0; - else - --Extract right side from left side expression if there is any !- - --Remove first negation tag to see if there is double negation - l_left_side := regexp_replace(l_left_side,l_right_side_regex,'\2'); - end if; - - - --check that on right side there is no extra negation - if regexp_count(l_left_side,l_rigth_side_expression) > 0 then - l_result := 0; - end if; - - --Now process right side of string - if l_right_side is not null then - l_result := least(l_result,valid_tag_expression(l_right_side)); - else - l_result := 0; - end if; - else - --We just process single tag. - l_left_side := l_tags; - l_left_side := regexp_replace(l_left_side,l_right_side_regex,'\2'); - if regexp_count(l_left_side,l_rigth_side_expression) > 0 then - l_result := 0; - end if; - end if; - - return l_result; - end; - - procedure build_tag_expression_filter(a_tags in varchar2,a_expression_tab in out t_expression_tab,a_parent_id varchar2 default null) is - l_left_side_expression varchar2(10) := '[|&,]'; - l_left_side_regex varchar2(50) := '([^|&,]*)([|&,])(.*)'; - l_left_side varchar2(4000); - - l_rigth_side_expression varchar2(10) := '[!-]'; - l_right_side_regex varchar2(50) := '([!-])([^!-].*)'; - l_right_side varchar2(4000); - - l_tags varchar2(4000) := a_tags; - l_result number :=1; - l_expression_rec t_expression_rec; - - begin - if a_expression_tab is null then - a_expression_tab := t_expression_tab(); - end if; - - l_expression_rec.id := sys_guid(); - l_expression_rec.parent_id := a_parent_id; - - if instr(substr(l_tags,1,1),'(',1,1) + instr(substr(l_tags,-1,1),')',-1,1) = 2 then - - if regexp_count(l_tags,l_right_side_regex) = 1 then - l_expression_rec.negated :=1; - l_tags := trim (leading '!' from l_tags); - end if; - - l_expression_rec.left_bracket := 1; - l_tags := trim(leading '(' from l_tags); - l_expression_rec.right_bracket := 1; - l_tags := trim(trailing ')' from l_tags); - end if; - - - --Check if there are any left side operators for first in order from left to right - if regexp_count(l_tags,l_left_side_expression) > 0 then - --Extract left part of operator and remaining of string to right - - --if there are bracketc extract it and record it - - l_left_side := regexp_replace(l_tags,l_left_side_regex,'\1'); - l_expression_rec.log_operator := regexp_replace(l_tags,l_left_side_regex,'\2'); - l_right_side := regexp_replace(l_tags,l_left_side_regex,'\3'); - a_expression_tab.extend; - a_expression_tab(a_expression_tab.last) := l_expression_rec; - - build_tag_expression_filter(l_left_side,a_expression_tab,l_expression_rec.id); - build_tag_expression_filter(l_right_side,a_expression_tab,l_expression_rec.id); - - else - if instr(substr(l_tags,1,1),'(',1,1) + instr(substr(l_tags,-1,1),')',-1,1) = 2 then - - if regexp_count(l_tags,l_right_side_regex) = 1 then - l_expression_rec.negated :=1; - l_tags := trim (leading '!' from l_tags); - end if; - - l_expression_rec.left_bracket := 1; - l_tags := trim(leading '(' from l_tags); - l_expression_rec.right_bracket := 1; - l_tags := trim(trailing ')' from l_tags); - end if; - l_expression_rec.expression := l_tags; - a_expression_tab.extend; - a_expression_tab(a_expression_tab.last) := l_expression_rec; - end if; - - end; - /* https://stackoverflow.com/questions/29634992/shunting-yard-validate-expression */ @@ -1251,9 +1113,9 @@ create or replace package body ut_utils is end loop; return l_infix_stack.pop; - end; + end convert_postfix_to_infix; - function convert_postfix_to_infix_where_sql(a_postfix_exp in ut_varchar2_list) + function conv_postfix_to_infix_sql(a_postfix_exp in ut_varchar2_list) return varchar2 is l_infix_stack ut_stack := ut_stack(); l_right_side varchar2(32767); @@ -1285,7 +1147,7 @@ create or replace package body ut_utils is end loop; return l_infix_stack.pop; - end; + end conv_postfix_to_infix_sql; begin --Define operator precedence diff --git a/source/core/ut_utils.pks b/source/core/ut_utils.pks index 2212454f4..4bc35466d 100644 --- a/source/core/ut_utils.pks +++ b/source/core/ut_utils.pks @@ -477,16 +477,6 @@ create or replace package ut_utils authid definer is */ function interval_to_text(a_interval yminterval_unconstrained) return varchar2; - /* - * Return number 1 or 0 if the list of tags is valid expression - */ - function valid_tag_expression(a_tags in varchar2) return number; - - /* - * Return number 1 or 0 if the list of tags is valid expression - */ - procedure build_tag_expression_filter(a_tags in varchar2,a_expression_tab in out t_expression_tab,a_parent_id varchar2 default null); - /* * Function that uses Dijkstra algorithm to parse mathematical and logical expression * and return a list of elements in Reverse Polish Notation ( postfix ) @@ -505,7 +495,7 @@ create or replace package ut_utils authid definer is * Function that converts postfix notation into infix and creating a string of sql filter * that checking a tags collections for tags according to posted logic. */ - function convert_postfix_to_infix_where_sql(a_postfix_exp in ut_varchar2_list) return varchar2; + function conv_postfix_to_infix_sql(a_postfix_exp in ut_varchar2_list) return varchar2; end ut_utils; / From 84e8684004ddbcc95318e24903fae9fd046bf12a Mon Sep 17 00:00:00 2001 From: Lukasz Wasylow Date: Mon, 10 Apr 2023 13:29:07 -0700 Subject: [PATCH 35/77] Update tests after removed function --- test/ut3_tester/core/test_ut_utils.pkb | 27 -------------------------- test/ut3_tester/core/test_ut_utils.pks | 3 --- 2 files changed, 30 deletions(-) diff --git a/test/ut3_tester/core/test_ut_utils.pkb b/test/ut3_tester/core/test_ut_utils.pkb index 8a497b9f1..ec7e4f403 100644 --- a/test/ut3_tester/core/test_ut_utils.pkb +++ b/test/ut3_tester/core/test_ut_utils.pkb @@ -489,32 +489,5 @@ end; ut.expect(l_expected).to_equal(l_actual); end; - procedure valid_tag_expressions is - begin - ut.expect(1).to_equal(ut3_develop.ut_utils.valid_tag_expression('tag1')); - ut.expect(1).to_equal(ut3_develop.ut_utils.valid_tag_expression('tag1|tag2')); - ut.expect(1).to_equal(ut3_develop.ut_utils.valid_tag_expression('tag1&tag2')); - ut.expect(1).to_equal(ut3_develop.ut_utils.valid_tag_expression('!tag1')); - ut.expect(1).to_equal(ut3_develop.ut_utils.valid_tag_expression('tag1|!tag2')); - ut.expect(1).to_equal(ut3_develop.ut_utils.valid_tag_expression('tag1&!tag2')); - ut.expect(1).to_equal(ut3_develop.ut_utils.valid_tag_expression('!tag1|!tag2')); - ut.expect(1).to_equal(ut3_develop.ut_utils.valid_tag_expression('!tag1&!tag2')); - - ut.expect(1).to_equal(ut3_develop.ut_utils.valid_tag_expression('tag1,tag2')); - ut.expect(1).to_equal(ut3_develop.ut_utils.valid_tag_expression('-tag1')); - ut.expect(1).to_equal(ut3_develop.ut_utils.valid_tag_expression('tag1,-tag2')); - ut.expect(1).to_equal(ut3_develop.ut_utils.valid_tag_expression('-tag1,-tag2')); - - ut.expect(1).to_equal(ut3_develop.ut_utils.valid_tag_expression('(!tag1|!tag2)|tag3')); - ut.expect(1).to_equal(ut3_develop.ut_utils.valid_tag_expression('(!tag1&!tag2)|(tag3&tag4)')); - - ut.expect(0).to_equal(ut3_develop.ut_utils.valid_tag_expression('tag1|')); - ut.expect(0).to_equal(ut3_develop.ut_utils.valid_tag_expression('&!tag2')); - ut.expect(0).to_equal(ut3_develop.ut_utils.valid_tag_expression('!!tag1|!tag2')); - ut.expect(0).to_equal(ut3_develop.ut_utils.valid_tag_expression('!tag1&!tag2|')); - ut.expect(0).to_equal(ut3_develop.ut_utils.valid_tag_expression('((!tag1|!tag2)|tag3')); - - end; - end test_ut_utils; / diff --git a/test/ut3_tester/core/test_ut_utils.pks b/test/ut3_tester/core/test_ut_utils.pks index 0f8545a45..4d83b5042 100644 --- a/test/ut3_tester/core/test_ut_utils.pks +++ b/test/ut3_tester/core/test_ut_utils.pks @@ -157,8 +157,5 @@ create or replace package test_ut_utils is --%endcontext - --%test(Test to validate different type of expressions passed as tags) - procedure valid_tag_expressions; - end test_ut_utils; / From 2e7a766f6169d0ad104448166ff3d0c84280b3e3 Mon Sep 17 00:00:00 2001 From: Lukasz Wasylow Date: Mon, 10 Apr 2023 13:57:54 -0700 Subject: [PATCH 36/77] Tidy up tests --- source/core/ut_suite_cache_manager.pkb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/core/ut_suite_cache_manager.pkb b/source/core/ut_suite_cache_manager.pkb index aea8c667e..a9153c7bd 100644 --- a/source/core/ut_suite_cache_manager.pkb +++ b/source/core/ut_suite_cache_manager.pkb @@ -268,7 +268,7 @@ create or replace package body ut_suite_cache_manager is if instr(l_tags,',') > 0 or instr(l_tags,'-') > 0 then l_tags := replace(replace_legacy_tag_notation(l_tags),' '); end if; - l_tags := ut_utils.convert_postfix_to_infix(ut_utils.shunt_logical_expression(l_tags)); + l_tags := ut_utils.conv_postfix_to_infix_sql(ut_utils.shunt_logical_expression(l_tags)); l_tags := REPLACE(l_tags, '|',' or '); l_tags := REPLACE(l_tags ,'&',' and '); l_tags := REPLACE(l_tags ,'!','not'); From cbdf83aee9491bd39e4bc06f74f72745fd5a6ef3 Mon Sep 17 00:00:00 2001 From: Lukasz Wasylow Date: Mon, 10 Apr 2023 14:02:57 -0700 Subject: [PATCH 37/77] Added ut_stack to uninstall --- source/uninstall_objects.sql | 2 ++ 1 file changed, 2 insertions(+) diff --git a/source/uninstall_objects.sql b/source/uninstall_objects.sql index 9531736b9..1e21f0f95 100644 --- a/source/uninstall_objects.sql +++ b/source/uninstall_objects.sql @@ -267,6 +267,8 @@ end; drop package ut_utils; +drop type ut_stack force; + drop sequence ut_savepoint_seq; drop type ut_documentation_reporter force; From 436eb5bed919df6b512688511a6d121a71c3a320 Mon Sep 17 00:00:00 2001 From: Lukasz Wasylow Date: Mon, 10 Apr 2023 18:00:07 -0700 Subject: [PATCH 38/77] Addressing test failures and sonar smells --- source/core/ut_utils.pkb | 51 +++++++++++++++----------- test/ut3_tester/core/test_ut_utils.pkb | 35 ++++++++++++++++++ test/ut3_tester/core/test_ut_utils.pks | 6 +++ 3 files changed, 71 insertions(+), 21 deletions(-) diff --git a/source/core/ut_utils.pkb b/source/core/ut_utils.pkb index 028a0c756..ac5f25645 100644 --- a/source/core/ut_utils.pkb +++ b/source/core/ut_utils.pkb @@ -1011,15 +1011,17 @@ create or replace package body ut_utils is l_token varchar2(32767); l_expect_operand boolean := true; l_expect_operator boolean := false; + l_idx pls_integer; begin --Tokenize a string into operators and tags select regexp_substr(l_tags,'([^!()|&]+)|([!()|&])', 1, level) as string_parts bulk collect into l_input_tokens from dual connect by regexp_substr (l_tags, '([^!()|&]+)|([!()|&])', 1, level) is not null; + l_idx := l_input_tokens.first; --Exuecute modified shunting algorithm - for token in 1..l_input_tokens.count loop - l_token := l_input_tokens(token); + WHILE (l_idx is not null) loop + l_token := l_input_tokens(l_idx); if (l_token member of gc_operators and l_token member of gc_binary_operator) then if not(l_expect_operator) then raise ex_invalid_tag_expression; @@ -1028,21 +1030,21 @@ create or replace package body ut_utils is l_rnp_tokens.extend; l_rnp_tokens(l_rnp_tokens.last) := l_operator_stack.pop; end loop; - l_operator_stack.push(l_input_tokens(token)); + l_operator_stack.push(l_input_tokens(l_idx)); l_expect_operand := true; l_expect_operator:= false; elsif (l_token member of gc_operators and l_token member of gc_unary_operator) then if not(l_expect_operand) then raise ex_invalid_tag_expression; end if; - l_operator_stack.push(l_input_tokens(token)); + l_operator_stack.push(l_input_tokens(l_idx)); l_expect_operand := true; l_expect_operator:= false; elsif l_token = '(' then if not(l_expect_operand) then raise ex_invalid_tag_expression; end if; - l_operator_stack.push(l_input_tokens(token)); + l_operator_stack.push(l_input_tokens(l_idx)); l_expect_operand := true; l_expect_operator:= false; elsif l_token = ')' then @@ -1066,6 +1068,7 @@ create or replace package body ut_utils is l_expect_operand := false; end if; + l_idx := l_input_tokens.next(l_idx); end loop; while l_operator_stack.top > 0 loop @@ -1091,25 +1094,28 @@ create or replace package body ut_utils is l_right_side varchar2(32767); l_left_side varchar2(32767); l_infix_exp varchar2(32767); + l_idx pls_integer; begin - for i in 1..a_postfix_exp.count loop + l_idx := a_postfix_exp.first; + while (l_idx is not null) loop --If token is operand but also single tag - if a_postfix_exp(i) not member of gc_operators then --its operand - l_infix_stack.push(a_postfix_exp(i)); + if a_postfix_exp(l_idx) not member of gc_operators then --its operand + l_infix_stack.push(a_postfix_exp(l_idx)); --If token is unary operator not - elsif a_postfix_exp(i) member of gc_unary_operator then + elsif a_postfix_exp(l_idx) member of gc_unary_operator then l_right_side := l_infix_stack.pop; - l_infix_exp := '('||a_postfix_exp(i)||l_right_side||')'; + l_infix_exp := '('||a_postfix_exp(l_idx)||l_right_side||')'; l_infix_stack.push(l_infix_exp); --If token is binary operator - elsif a_postfix_exp(i) member of gc_binary_operator then + elsif a_postfix_exp(l_idx) member of gc_binary_operator then l_right_side := l_infix_stack.pop; l_left_side := l_infix_stack.pop; - l_infix_exp := '('||l_left_side||a_postfix_exp(i)||l_right_side||')'; + l_infix_exp := '('||l_left_side||a_postfix_exp(l_idx)||l_right_side||')'; l_infix_stack.push(l_infix_exp); else null; end if; + l_idx := a_postfix_exp.next(l_idx); end loop; return l_infix_stack.pop; @@ -1122,28 +1128,31 @@ create or replace package body ut_utils is l_left_side varchar2(32767); l_infix_exp varchar2(32767); l_member_token varchar2(20) := ' member of tags'; + l_idx pls_integer; begin - for i in 1..a_postfix_exp.count loop + l_idx := a_postfix_exp.first; + while ( l_idx is not null) loop --If token is operand but also single tag - if regexp_count(a_postfix_exp(i),'[!()|&]') = 0 then - l_infix_stack.push(q'[']'||a_postfix_exp(i)||q'[']'||l_member_token); + if regexp_count(a_postfix_exp(l_idx),'[!()|&]') = 0 then + l_infix_stack.push(q'[']'||a_postfix_exp(l_idx)||q'[']'||l_member_token); --If token is operand but containing other expressions - elsif a_postfix_exp(i) not member of gc_operators then - l_infix_stack.push(a_postfix_exp(i)); + elsif a_postfix_exp(l_idx) not member of gc_operators then + l_infix_stack.push(a_postfix_exp(l_idx)); --If token is unary operator not - elsif a_postfix_exp(i) member of gc_unary_operator then + elsif a_postfix_exp(l_idx) member of gc_unary_operator then l_right_side := l_infix_stack.pop; - l_infix_exp := a_postfix_exp(i)||'('||l_right_side||')'; + l_infix_exp := a_postfix_exp(l_idx)||'('||l_right_side||')'; l_infix_stack.push(l_infix_exp); --If token is binary operator - elsif a_postfix_exp(i) member of gc_binary_operator then + elsif a_postfix_exp(l_idx) member of gc_binary_operator then l_right_side := l_infix_stack.pop; l_left_side := l_infix_stack.pop; - l_infix_exp := '('||l_left_side||a_postfix_exp(i)||l_right_side||')'; + l_infix_exp := '('||l_left_side||a_postfix_exp(l_idx)||l_right_side||')'; l_infix_stack.push(l_infix_exp); else null; end if; + l_idx := a_postfix_exp.next(l_idx); end loop; return l_infix_stack.pop; diff --git a/test/ut3_tester/core/test_ut_utils.pkb b/test/ut3_tester/core/test_ut_utils.pkb index ec7e4f403..1f64c621e 100644 --- a/test/ut3_tester/core/test_ut_utils.pkb +++ b/test/ut3_tester/core/test_ut_utils.pkb @@ -489,5 +489,40 @@ end; ut.expect(l_expected).to_equal(l_actual); end; + + procedure test_conversion_to_rpn is + l_postfix ut3_develop.ut_varchar2_list; + l_postfix_string varchar2(4000); + begin + l_postfix := ut3_develop.ut_utils.shunt_logical_expression('A'); + l_postfix_string := ut3_develop.ut_utils.table_to_clob(l_postfix,''); + ut.expect(l_postfix_string).to_equal('A'); + + l_postfix := ut3_develop.ut_utils.shunt_logical_expression('A|B'); + l_postfix_string := ut3_develop.ut_utils.table_to_clob(l_postfix,''); + ut.expect(l_postfix_string).to_equal('AB|'); + + l_postfix := ut3_develop.ut_utils.shunt_logical_expression('(a|b)|c&d'); + l_postfix_string := ut3_develop.ut_utils.table_to_clob(l_postfix,''); + ut.expect(l_postfix_string).to_equal('ab|cd&|'); + end; + + procedure test_conversion_from_rpn_to_infix is + l_postfix_rpn ut3_develop.ut_varchar2_list; + l_infix_string varchar2(4000); + begin + l_postfix_rpn := ut3_develop.ut_varchar2_list('A'); + l_infix_string := ut3_develop.ut_utils.convert_postfix_to_infix(l_postfix_rpn); + ut.expect(l_infix_string).to_equal('A'); + + l_postfix_rpn := ut3_develop.ut_varchar2_list('A','B','|'); + l_infix_string := ut3_develop.ut_utils.convert_postfix_to_infix(l_postfix_rpn); + ut.expect(l_infix_string).to_equal('(A|B)'); + + l_postfix_rpn := ut3_develop.ut_varchar2_list('a','b','|','c','d','&','|'); + l_infix_string := ut3_develop.ut_utils.convert_postfix_to_infix(l_postfix_rpn); + ut.expect(l_infix_string).to_equal('((a|b)|(c&d))'); + end; + end test_ut_utils; / diff --git a/test/ut3_tester/core/test_ut_utils.pks b/test/ut3_tester/core/test_ut_utils.pks index 4d83b5042..7bfd9b36e 100644 --- a/test/ut3_tester/core/test_ut_utils.pks +++ b/test/ut3_tester/core/test_ut_utils.pks @@ -156,6 +156,12 @@ create or replace package test_ut_utils is --%endcontext + + --%test( Test conversion of expression into Reverse Polish Notation) + procedure test_conversion_to_rpn; + + --%test( Test conversion of expression from Reverse Polish Notation into infix) + procedure test_conversion_from_rpn_to_infix; end test_ut_utils; / From 3d77514448e9720ad11589343d171723ad8f1c72 Mon Sep 17 00:00:00 2001 From: Lukasz Wasylow Date: Mon, 10 Apr 2023 18:08:42 -0700 Subject: [PATCH 39/77] Update name --- test/ut3_tester/core/test_ut_utils.pkb | 2 +- test/ut3_tester/core/test_ut_utils.pks | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/test/ut3_tester/core/test_ut_utils.pkb b/test/ut3_tester/core/test_ut_utils.pkb index 1f64c621e..126ada89c 100644 --- a/test/ut3_tester/core/test_ut_utils.pkb +++ b/test/ut3_tester/core/test_ut_utils.pkb @@ -507,7 +507,7 @@ end; ut.expect(l_postfix_string).to_equal('ab|cd&|'); end; - procedure test_conversion_from_rpn_to_infix is + procedure test_conv_from_rpn_to_infix is l_postfix_rpn ut3_develop.ut_varchar2_list; l_infix_string varchar2(4000); begin diff --git a/test/ut3_tester/core/test_ut_utils.pks b/test/ut3_tester/core/test_ut_utils.pks index 7bfd9b36e..6d11f65d1 100644 --- a/test/ut3_tester/core/test_ut_utils.pks +++ b/test/ut3_tester/core/test_ut_utils.pks @@ -161,7 +161,7 @@ create or replace package test_ut_utils is procedure test_conversion_to_rpn; --%test( Test conversion of expression from Reverse Polish Notation into infix) - procedure test_conversion_from_rpn_to_infix; + procedure test_conv_from_rpn_to_infix; end test_ut_utils; / From bf6959fc63c4c36ff24f62d28e8a5b1d6b323ce4 Mon Sep 17 00:00:00 2001 From: Lukasz Wasylow Date: Mon, 10 Apr 2023 20:39:25 -0700 Subject: [PATCH 40/77] Update tests and code --- source/core/ut_utils.pkb | 30 +++++++++++--------------- source/core/ut_utils.pks | 7 ++++-- test/ut3_tester/core/test_ut_utils.pkb | 29 +++++++++++++++++++++++-- test/ut3_tester/core/test_ut_utils.pks | 3 +++ test/ut3_user/api/test_ut_run.pkb | 15 ++++++++++++- 5 files changed, 62 insertions(+), 22 deletions(-) diff --git a/source/core/ut_utils.pkb b/source/core/ut_utils.pkb index ac5f25645..1c99a3a42 100644 --- a/source/core/ut_utils.pkb +++ b/source/core/ut_utils.pkb @@ -1000,24 +1000,30 @@ create or replace package body ut_utils is return l_result; end; + function tokenize_tags_string(a_tags in varchar2) return ut_varchar2_list is + l_tags_tokens ut_varchar2_list := ut_varchar2_list(); + begin + --Tokenize a string into operators and tags + select regexp_substr(a_tags,'([^!()|&]+)|([!()|&])', 1, level) as string_parts + bulk collect into l_tags_tokens + from dual connect by regexp_substr (a_tags, '([^!()|&]+)|([!()|&])', 1, level) is not null; + + return l_tags_tokens; + end; + /* https://stackoverflow.com/questions/29634992/shunting-yard-validate-expression */ function shunt_logical_expression(a_tags in varchar2) return ut_varchar2_list is l_tags varchar2(32767) := a_tags; l_operator_stack ut_stack := ut_stack(); - l_input_tokens ut_varchar2_list := ut_varchar2_list(); + l_input_tokens ut_varchar2_list := tokenize_tags_string(a_tags); l_rnp_tokens ut_varchar2_list := ut_varchar2_list(); l_token varchar2(32767); l_expect_operand boolean := true; l_expect_operator boolean := false; l_idx pls_integer; begin - --Tokenize a string into operators and tags - select regexp_substr(l_tags,'([^!()|&]+)|([!()|&])', 1, level) as string_parts - bulk collect into l_input_tokens - from dual connect by regexp_substr (l_tags, '([^!()|&]+)|([!()|&])', 1, level) is not null; - l_idx := l_input_tokens.first; --Exuecute modified shunting algorithm WHILE (l_idx is not null) loop @@ -1071,7 +1077,7 @@ create or replace package body ut_utils is l_idx := l_input_tokens.next(l_idx); end loop; - while l_operator_stack.top > 0 loop + while l_operator_stack.peek is not null loop if l_operator_stack.peek in ('(',')') then raise ex_invalid_tag_expression; end if; @@ -1082,12 +1088,6 @@ create or replace package body ut_utils is return l_rnp_tokens; end shunt_logical_expression; - procedure shunt_logical_expression(a_tags in varchar2) is - a_postfix ut_varchar2_list; - begin - a_postfix := ut_utils.shunt_logical_expression(a_tags); - end shunt_logical_expression; - function convert_postfix_to_infix(a_postfix_exp in ut_varchar2_list) return varchar2 is l_infix_stack ut_stack := ut_stack(); @@ -1112,8 +1112,6 @@ create or replace package body ut_utils is l_left_side := l_infix_stack.pop; l_infix_exp := '('||l_left_side||a_postfix_exp(l_idx)||l_right_side||')'; l_infix_stack.push(l_infix_exp); - else - null; end if; l_idx := a_postfix_exp.next(l_idx); end loop; @@ -1149,8 +1147,6 @@ create or replace package body ut_utils is l_left_side := l_infix_stack.pop; l_infix_exp := '('||l_left_side||a_postfix_exp(l_idx)||l_right_side||')'; l_infix_stack.push(l_infix_exp); - else - null; end if; l_idx := a_postfix_exp.next(l_idx); end loop; diff --git a/source/core/ut_utils.pks b/source/core/ut_utils.pks index 4bc35466d..67b09b2f9 100644 --- a/source/core/ut_utils.pks +++ b/source/core/ut_utils.pks @@ -476,6 +476,11 @@ create or replace package ut_utils authid definer is * Return value of interval in plain english */ function interval_to_text(a_interval yminterval_unconstrained) return varchar2; + + /* + * Return table of tokens character by character + */ + function tokenize_tags_string(a_tags in varchar2) return ut_varchar2_list; /* * Function that uses Dijkstra algorithm to parse mathematical and logical expression @@ -484,8 +489,6 @@ create or replace package ut_utils authid definer is */ function shunt_logical_expression(a_tags in varchar2) return ut_varchar2_list; - procedure shunt_logical_expression(a_tags in varchar2); - /* * Function that converts postfix notation into infix */ diff --git a/test/ut3_tester/core/test_ut_utils.pkb b/test/ut3_tester/core/test_ut_utils.pkb index 126ada89c..22733c4ce 100644 --- a/test/ut3_tester/core/test_ut_utils.pkb +++ b/test/ut3_tester/core/test_ut_utils.pkb @@ -504,7 +504,11 @@ end; l_postfix := ut3_develop.ut_utils.shunt_logical_expression('(a|b)|c&d'); l_postfix_string := ut3_develop.ut_utils.table_to_clob(l_postfix,''); - ut.expect(l_postfix_string).to_equal('ab|cd&|'); + ut.expect(l_postfix_string).to_equal('ab|cd&|'); + + l_postfix := ut3_develop.ut_utils.shunt_logical_expression('!a|b'); + l_postfix_string := ut3_develop.ut_utils.table_to_clob(l_postfix,''); + ut.expect(l_postfix_string).to_equal('a!b|'); end; procedure test_conv_from_rpn_to_infix is @@ -521,7 +525,28 @@ end; l_postfix_rpn := ut3_develop.ut_varchar2_list('a','b','|','c','d','&','|'); l_infix_string := ut3_develop.ut_utils.convert_postfix_to_infix(l_postfix_rpn); - ut.expect(l_infix_string).to_equal('((a|b)|(c&d))'); + ut.expect(l_infix_string).to_equal('((a|b)|(c&d))'); + + l_postfix_rpn := ut3_develop.ut_varchar2_list('a','b','!','|'); + l_infix_string := ut3_develop.ut_utils.convert_postfix_to_infix(l_postfix_rpn); + ut.expect(l_infix_string).to_equal('(a|(!b))'); + end; + + procedure conv_from_rpn_to_sql_filter is + l_postfix_rpn ut3_develop.ut_varchar2_list; + l_infix_string varchar2(4000); + begin + l_postfix_rpn := ut3_develop.ut_varchar2_list('A'); + l_infix_string := ut3_develop.ut_utils.conv_postfix_to_infix_sql(l_postfix_rpn); + ut.expect(l_infix_string).to_equal(q'['A' member of tags]'); + + l_postfix_rpn := ut3_develop.ut_varchar2_list('A','B','|'); + l_infix_string := ut3_develop.ut_utils.conv_postfix_to_infix_sql(l_postfix_rpn); + ut.expect(l_infix_string).to_equal(q'[('A' member of tags|'B' member of tags)]'); + + l_postfix_rpn := ut3_develop.ut_varchar2_list('a','b','!','|'); + l_infix_string := ut3_develop.ut_utils.conv_postfix_to_infix_sql(l_postfix_rpn); + ut.expect(l_infix_string).to_equal(q'[('a' member of tags|!('b' member of tags))]'); end; end test_ut_utils; diff --git a/test/ut3_tester/core/test_ut_utils.pks b/test/ut3_tester/core/test_ut_utils.pks index 6d11f65d1..1561e1136 100644 --- a/test/ut3_tester/core/test_ut_utils.pks +++ b/test/ut3_tester/core/test_ut_utils.pks @@ -163,5 +163,8 @@ create or replace package test_ut_utils is --%test( Test conversion of expression from Reverse Polish Notation into infix) procedure test_conv_from_rpn_to_infix; + --%test( Test conversion of expression from Reverse Polish Notation into custom where filter for SQL) + procedure conv_from_rpn_to_sql_filter; + end test_ut_utils; / diff --git a/test/ut3_user/api/test_ut_run.pkb b/test/ut3_user/api/test_ut_run.pkb index 7eb377f4b..25b50343e 100644 --- a/test/ut3_user/api/test_ut_run.pkb +++ b/test/ut3_user/api/test_ut_run.pkb @@ -1260,7 +1260,20 @@ procedure tag_exclude_run_fun_pth_lst_lg is l_results := ut3_tester_helper.run_helper.run(a_tags => '(!development&end_to_end|)'); ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_match('^\s*invalid_tag_expression \[[,\.0-9]+ sec\]\s*$','m'); - ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like('%(FAILED -%'); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like('%(FAILED -%'); + + l_results := ut3_tester_helper.run_helper.run(a_tags => '(!development&!!end_to_end)'); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_match('^\s*invalid_tag_expression \[[,\.0-9]+ sec\]\s*$','m'); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like('%(FAILED -%'); + + l_results := ut3_tester_helper.run_helper.run(a_tags => '(&development&end_to_end)'); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_match('^\s*invalid_tag_expression \[[,\.0-9]+ sec\]\s*$','m'); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like('%(FAILED -%'); + + l_results := ut3_tester_helper.run_helper.run(a_tags => '(development|end_to_end))'); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_match('^\s*invalid_tag_expression \[[,\.0-9]+ sec\]\s*$','m'); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like('%(FAILED -%'); + end; procedure set_application_info is From d8233ff6587f81c0a0a6430a4f33747bc2723983 Mon Sep 17 00:00:00 2001 From: Lukasz Wasylow Date: Tue, 11 Apr 2023 20:33:04 -0700 Subject: [PATCH 41/77] fixing typo in docs --- docs/userguide/annotations.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/userguide/annotations.md b/docs/userguide/annotations.md index 4017521b5..8e5a5d1ca 100644 --- a/docs/userguide/annotations.md +++ b/docs/userguide/annotations.md @@ -1684,7 +1684,7 @@ Execution of the test is done by using the parameter `a_tags` with tag expressio ```sql linenums="1" -select * from table(ut.run(a_tags => 'fast||!complex')); +select * from table(ut.run(a_tags => 'fast|!complex')); ``` The above call will execute all tests from `ut_sample_test` package as the whole suite is tagged with `api` because a suite meet expression condition. From bd860f602d458575d00010834e5ca28058ec09cb Mon Sep 17 00:00:00 2001 From: Lukasz Wasylow Date: Tue, 11 Apr 2023 20:55:34 -0700 Subject: [PATCH 42/77] Removed unused variable --- source/core/ut_utils.pkb | 1 - 1 file changed, 1 deletion(-) diff --git a/source/core/ut_utils.pkb b/source/core/ut_utils.pkb index 1c99a3a42..a8528fa86 100644 --- a/source/core/ut_utils.pkb +++ b/source/core/ut_utils.pkb @@ -1015,7 +1015,6 @@ create or replace package body ut_utils is https://stackoverflow.com/questions/29634992/shunting-yard-validate-expression */ function shunt_logical_expression(a_tags in varchar2) return ut_varchar2_list is - l_tags varchar2(32767) := a_tags; l_operator_stack ut_stack := ut_stack(); l_input_tokens ut_varchar2_list := tokenize_tags_string(a_tags); l_rnp_tokens ut_varchar2_list := ut_varchar2_list(); From e6a6a4cbf108b44932fb114c07e2d18f62ee687f Mon Sep 17 00:00:00 2001 From: Jacek Gebal Date: Wed, 12 Apr 2023 19:54:56 +0300 Subject: [PATCH 43/77] Improvements to teamcity reporter (used by DataGrip and IntelliJ - JetBrains IDEs) The reporter will now report all errors and all failed expectations. The test description will be used instead of test name. --- source/reporters/ut_teamcity_reporter.tpb | 33 +++++++++------- .../reporters/ut_teamcity_reporter_helper.pkb | 2 - .../reporters/test_teamcity_reporter.pkb | 39 +++++++++++++++++-- .../reporters/test_teamcity_reporter.pks | 3 ++ 4 files changed, 58 insertions(+), 19 deletions(-) diff --git a/source/reporters/ut_teamcity_reporter.tpb b/source/reporters/ut_teamcity_reporter.tpb index 95bea32b8..e05dc994f 100644 --- a/source/reporters/ut_teamcity_reporter.tpb +++ b/source/reporters/ut_teamcity_reporter.tpb @@ -108,23 +108,28 @@ create or replace type body ut_teamcity_reporter is ut_teamcity_reporter_helper.test_failed( a_test_name => l_test_full_name, a_msg => 'Error occured', - a_details => - trim(l_std_err_msg) - || case when a_test.failed_expectations is not null - and a_test.failed_expectations.count>0 - then a_test.failed_expectations(1).message end + a_details => trim(l_std_err_msg) ) ); + for i in 1 .. a_test.failed_expectations.count loop + ut_utils.append_to_list( + l_results, + ut_teamcity_reporter_helper.test_failed( + a_test_name => l_test_full_name, + a_msg => a_test.failed_expectations(i).description, + a_details => a_test.failed_expectations(i).message ) + ); + end loop; elsif a_test.failed_expectations is not null and a_test.failed_expectations.count > 0 then - -- Teamcity supports only a single failure message - - ut_utils.append_to_list( - l_results, - ut_teamcity_reporter_helper.test_failed( - a_test_name => l_test_full_name, - a_msg => a_test.failed_expectations(a_test.failed_expectations.first).description, - a_details => a_test.failed_expectations(a_test.failed_expectations.first).message ) - ); + for i in 1 .. a_test.failed_expectations.count loop + ut_utils.append_to_list( + l_results, + ut_teamcity_reporter_helper.test_failed( + a_test_name => l_test_full_name, + a_msg => a_test.failed_expectations(i).description, + a_details => a_test.failed_expectations(i).message ) + ); + end loop; elsif a_test.result = ut_utils.gc_failure then ut_utils.append_to_list( l_results, diff --git a/source/reporters/ut_teamcity_reporter_helper.pkb b/source/reporters/ut_teamcity_reporter_helper.pkb index 84a9d19e8..1633d10aa 100644 --- a/source/reporters/ut_teamcity_reporter_helper.pkb +++ b/source/reporters/ut_teamcity_reporter_helper.pkb @@ -73,8 +73,6 @@ create or replace package body ut_teamcity_reporter_helper is 'true' when false then 'false' - else - null end; l_props('flowId') := a_flow_id; return message('testStarted', l_props); diff --git a/test/ut3_user/reporters/test_teamcity_reporter.pkb b/test/ut3_user/reporters/test_teamcity_reporter.pkb index ff550e488..cf7a925a1 100644 --- a/test/ut3_user/reporters/test_teamcity_reporter.pkb +++ b/test/ut3_user/reporters/test_teamcity_reporter.pkb @@ -31,12 +31,26 @@ create or replace package body test_teamcity_reporter as end; end;]'; + execute immediate q'[create or replace package check_multiple_failures is + --%suite + + --%test + procedure multi_failure; + end;]'; + execute immediate q'[create or replace package body check_multiple_failures is + procedure multi_failure is + begin + ut3_develop.ut.expect(1).to_be_null; + ut3_develop.ut.expect(2).to_equal(1); + ut3_develop.ut.expect('Bad').to_equal('Good'); + end; + end;]'; + end; procedure report_produces_expected_out is l_output_data ut3_develop.ut_varchar2_list; - l_output clob; l_expected varchar2(32767); begin l_expected := q'{%##teamcity[testSuiteStarted timestamp='%' name='org'] @@ -84,7 +98,6 @@ create or replace package body test_teamcity_reporter as procedure escape_special_chars is l_output_data ut3_develop.ut_varchar2_list; - l_output clob; l_expected varchar2(32767); begin l_expected := q'{%##teamcity[testSuiteStarted timestamp='%' name='A suite with |'quote|''] @@ -103,7 +116,6 @@ create or replace package body test_teamcity_reporter as procedure trims_long_output is l_output_data ut3_develop.ut_varchar2_list; - l_output clob; l_expected varchar2(32767); begin l_expected := q'{%##teamcity[testSuiteStarted timestamp='%' name='check_trims_long_output'] @@ -120,11 +132,32 @@ create or replace package body test_teamcity_reporter as ut.expect(ut3_tester_helper.main_helper.table_to_clob(l_output_data)).to_be_like(l_expected); end; + procedure report_mutiple_expectations is + l_output_data ut3_develop.ut_varchar2_list; + l_expected varchar2(32767); + begin + l_expected := q'{%##teamcity[testSuiteStarted timestamp='%' name='check_multiple_failures'] +%##teamcity[testStarted timestamp='%' captureStandardOutput='true' name='ut3_user.check_multiple_failures.multi_failure'] +%##teamcity[testFailed timestamp='%' details='Actual: 1 (number) was expected to be null' name='ut3_user.check_multiple_failures.multi_failure'] +%##teamcity[testFailed timestamp='%' details='Actual: 2 (number) was expected to equal: 1 (number)' name='ut3_user.check_multiple_failures.multi_failure'] +%##teamcity[testFailed timestamp='%' details='Actual: |'Bad|' (varchar2) was expected to equal: |'Good|' (varchar2)' name='ut3_user.check_multiple_failures.multi_failure'] +%##teamcity[testFinished timestamp='%' duration='%' name='ut3_user.check_multiple_failures.multi_failure'] +%##teamcity[testSuiteFinished timestamp='%' name='check_multiple_failures']}'; + --act + select * + bulk collect into l_output_data + from table(ut3_develop.ut.run('check_multiple_failures',ut3_develop.ut_teamcity_reporter())); + + --assert + ut.expect(ut3_tester_helper.main_helper.table_to_clob(l_output_data)).to_be_like(l_expected); + end; + procedure remove_test_package is pragma autonomous_transaction; begin execute immediate 'drop package check_escape_special_chars'; execute immediate 'drop package check_trims_long_output'; + execute immediate 'drop package check_multiple_failures'; end; end; diff --git a/test/ut3_user/reporters/test_teamcity_reporter.pks b/test/ut3_user/reporters/test_teamcity_reporter.pks index f849751f1..30ae7e26b 100644 --- a/test/ut3_user/reporters/test_teamcity_reporter.pks +++ b/test/ut3_user/reporters/test_teamcity_reporter.pks @@ -15,6 +15,9 @@ create or replace package test_teamcity_reporter as --%test(Trims output so it fits into 4000 chars) procedure trims_long_output; + --%test(Reports failures on multiple expectations) + procedure report_mutiple_expectations; + --%afterall procedure remove_test_package; From 0497dcfadcac637d186fdbc0aa36338d178f597d Mon Sep 17 00:00:00 2001 From: Jacek Gebal Date: Wed, 12 Apr 2023 20:02:14 +0300 Subject: [PATCH 44/77] Adding Oracle 23-free-developer-edition --- .github/workflows/build.yml | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 0ec37efe2..c2fe3971e 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -58,6 +58,11 @@ jobs: oracle-sid: 'XE' oracle-version: "gvenzl/oracle-xe:21-slim" oracle-base: '/opt/oracle' + - id: 7 + db_version_name: '23free' + oracle-sid: 'FREEPDB1' + oracle-version: "gvenzl/oracle-free:23-slim" + oracle-base: '/opt/oracle' services: html_checker: From 55a2ecfcb73ff6389058423f41adf9e20279fc1c Mon Sep 17 00:00:00 2001 From: Jacek Gebal Date: Wed, 12 Apr 2023 21:00:57 +0300 Subject: [PATCH 45/77] Adjusted tests for ORacle 23c (change of error message for ORA-06502) --- test/ut3_user/reporters/test_documentation_reporter.pkb | 2 +- test/ut3_user/reporters/test_teamcity_reporter.pkb | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/test/ut3_user/reporters/test_documentation_reporter.pkb b/test/ut3_user/reporters/test_documentation_reporter.pkb index 6ace70c49..2a05679ff 100644 --- a/test/ut3_user/reporters/test_documentation_reporter.pkb +++ b/test/ut3_user/reporters/test_documentation_reporter.pkb @@ -37,7 +37,7 @@ Failures: % % 2) erroring_test - ORA-06502: PL/SQL: numeric or value error: character to number conversion error + ORA-06502: PL/SQL: %: character to number conversion error ORA-06512: at "UT3_USER.TEST_REPORTERS", line 44% ORA-06512: at line 6 Finished in % seconds diff --git a/test/ut3_user/reporters/test_teamcity_reporter.pkb b/test/ut3_user/reporters/test_teamcity_reporter.pkb index cf7a925a1..7c4c1e513 100644 --- a/test/ut3_user/reporters/test_teamcity_reporter.pkb +++ b/test/ut3_user/reporters/test_teamcity_reporter.pkb @@ -77,8 +77,8 @@ create or replace package body test_teamcity_reporter as -%##teamcity[testStdErr timestamp='%' name='ut3_user.test_reporters.erroring_test' out='Test exception:|nORA-06502: PL/SQL: numeric or value error: character to number conversion error|nORA-06512: at "UT3_USER.TEST_REPORTERS", line %|nORA-06512: at %|n'] -%##teamcity[testFailed timestamp='%' details='Test exception:|nORA-06502: PL/SQL: numeric or value error: character to number conversion error|nORA-06512: at "UT3_USER.TEST_REPORTERS", line %|nORA-06512: at %|n' message='Error occured' name='ut3_user.test_reporters.erroring_test'] +%##teamcity[testStdErr timestamp='%' name='ut3_user.test_reporters.erroring_test' out='Test exception:|nORA-06502: PL/SQL: %: character to number conversion error|nORA-06512: at "UT3_USER.TEST_REPORTERS", line %|nORA-06512: at %|n'] +%##teamcity[testFailed timestamp='%' details='Test exception:|nORA-06502: PL/SQL: %: character to number conversion error|nORA-06512: at "UT3_USER.TEST_REPORTERS", line %|nORA-06512: at %|n' message='Error occured' name='ut3_user.test_reporters.erroring_test'] %##teamcity[testFinished timestamp='%' duration='%' name='ut3_user.test_reporters.erroring_test'] %##teamcity[testStarted timestamp='%' captureStandardOutput='true' name='ut3_user.test_reporters.disabled_test'] %##teamcity[testIgnored timestamp='%' name='ut3_user.test_reporters.disabled_test'] From 313d5e9019f1253cc222821c802a11c8cdd7ed1a Mon Sep 17 00:00:00 2001 From: Lukasz Wasylow Date: Wed, 12 Apr 2023 18:40:28 -0700 Subject: [PATCH 46/77] Stage 1 Resolving PR comments --- source/api/ut_runner.pkb | 8 ++-- source/core/types/ut_stack.tpb | 2 +- source/core/types/ut_stack.tps | 2 +- source/core/ut_suite_builder.pkb | 2 +- source/core/ut_suite_cache_manager.pkb | 62 +++++++++++++------------- source/core/ut_utils.pkb | 55 +++++------------------ source/core/ut_utils.pks | 5 --- test/ut3_tester/core/test_ut_utils.pkb | 21 --------- test/ut3_tester/core/test_ut_utils.pks | 3 -- 9 files changed, 50 insertions(+), 110 deletions(-) diff --git a/source/api/ut_runner.pkb b/source/api/ut_runner.pkb index 3d4550cf9..3ec2a5393 100644 --- a/source/api/ut_runner.pkb +++ b/source/api/ut_runner.pkb @@ -78,9 +78,7 @@ create or replace package body ut_runner is l_run ut_run; l_coverage_schema_names ut_varchar2_rows; l_paths ut_varchar2_list; - l_random_test_order_seed positive; - l_tags varchar2(4000) := a_tags; - + l_random_test_order_seed positive; begin ut_event_manager.initialize(); if a_reporters is not empty then @@ -135,10 +133,10 @@ create or replace package body ut_runner is a_test_file_mappings => set(a_test_file_mappings), a_client_character_set => a_client_character_set, a_random_test_order_seed => l_random_test_order_seed, - a_run_tags => l_tags + a_run_tags => a_tags ); - ut_suite_manager.configure_execution_by_path(l_paths, l_run.items, l_random_test_order_seed, l_tags); + ut_suite_manager.configure_execution_by_path(l_paths, l_run.items, l_random_test_order_seed, a_tags); if a_force_manual_rollback then l_run.set_rollback_type( a_rollback_type => ut_utils.gc_rollback_manual, a_force => true ); end if; diff --git a/source/core/types/ut_stack.tpb b/source/core/types/ut_stack.tpb index a7f7ab0c2..b5f4e8747 100644 --- a/source/core/types/ut_stack.tpb +++ b/source/core/types/ut_stack.tpb @@ -1,7 +1,7 @@ create or replace type body ut_stack as /* utPLSQL - Version 3 - Copyright 2016 - 2021 utPLSQL Project + Copyright 2016 - 2023 utPLSQL Project Licensed under the Apache License, Version 2.0 (the "License"): you may not use this file except in compliance with the License. diff --git a/source/core/types/ut_stack.tps b/source/core/types/ut_stack.tps index 7b8c145c2..23112fdde 100644 --- a/source/core/types/ut_stack.tps +++ b/source/core/types/ut_stack.tps @@ -3,7 +3,7 @@ create or replace type ut_stack as object ( tokens ut_varchar2_list, /* utPLSQL - Version 3 - Copyright 2016 - 2021 utPLSQL Project + Copyright 2016 - 2023 utPLSQL Project Licensed under the Apache License, Version 2.0 (the "License"): you may not use this file except in compliance with the License. diff --git a/source/core/ut_suite_builder.pkb b/source/core/ut_suite_builder.pkb index 4aa2ff915..ebb113370 100644 --- a/source/core/ut_suite_builder.pkb +++ b/source/core/ut_suite_builder.pkb @@ -205,7 +205,7 @@ create or replace package body ut_suite_builder is l_tag_items := ut_utils.trim_list_elements(ut_utils.string_to_table(a_tags_ann_text(l_annotation_pos),',')); if l_tag_items is not empty then for i in 1 .. l_tag_items.count loop - if regexp_like(l_tag_items(i),'^[^-!&|](\S)+$') then + if regexp_like(l_tag_items(i),'^[^-](\S)+$') then l_tags_list.extend(); l_tags_list(l_tags_list.last) := l_tag_items(i); else diff --git a/source/core/ut_suite_cache_manager.pkb b/source/core/ut_suite_cache_manager.pkb index a9153c7bd..858069c8f 100644 --- a/source/core/ut_suite_cache_manager.pkb +++ b/source/core/ut_suite_cache_manager.pkb @@ -238,44 +238,46 @@ create or replace package body ut_suite_cache_manager is l_tags_exclude varchar2(4000); l_return_tag varchar2(4000); begin - select listagg( t.column_value,'|') - within group( order by column_value) - into l_tags_include - from table(l_tags) t - where t.column_value not like '-%'; - - select listagg( replace(t.column_value,'-','!'),' & ') - within group( order by column_value) - into l_tags_exclude - from table(l_tags) t - where t.column_value like '-%'; - - l_return_tag:= - case when l_tags_include is not null then - '('||l_tags_include||')' else null end || - case when l_tags_include is not null and l_tags_exclude is not null then - ' & ' else null end || - case when l_tags_exclude is not null then - '('||l_tags_exclude||')' else null end; + if instr(a_tags,',') > 0 or instr(a_tags,'-') > 0 then + + select '('||listagg( t.column_value,'|') + within group( order by column_value)||')' + into l_tags_include + from table(l_tags) t + where t.column_value not like '-%'; + select '('||listagg( replace(t.column_value,'-','!'),' & ') + within group( order by column_value)||')' + into l_tags_exclude + from table(l_tags) t + where t.column_value like '-%'; + + + l_return_tag:= + case + when l_tags_include <> '()' and l_tags_exclude <> '()' + then l_tags_include || ' & ' || l_tags_exclude + when l_tags_include <> '()' + then l_tags_include + when l_tags_exclude <> '()' + then l_tags_exclude + end; + else + l_return_tag := a_tags; + end if; return l_return_tag; end; function create_where_filter(a_tags varchar2 ) return varchar2 is - l_tags varchar2(4000):= replace(a_tags,' '); + l_tags varchar2(4000); begin - if instr(l_tags,',') > 0 or instr(l_tags,'-') > 0 then - l_tags := replace(replace_legacy_tag_notation(l_tags),' '); - end if; + l_tags := replace(replace_legacy_tag_notation(a_tags),' '); l_tags := ut_utils.conv_postfix_to_infix_sql(ut_utils.shunt_logical_expression(l_tags)); - l_tags := REPLACE(l_tags, '|',' or '); - l_tags := REPLACE(l_tags ,'&',' and '); - l_tags := REPLACE(l_tags ,'!','not'); - return l_tags; - exception - when ut_utils.ex_invalid_tag_expression then - raise_application_error(ut_utils.gc_invalid_tag_expression, 'Invalid Tag expression'); + l_tags := replace(l_tags, '|',' or '); + l_tags := replace(l_tags ,'&',' and '); + l_tags := replace(l_tags ,'!','not'); + return l_tags; end; /* diff --git a/source/core/ut_utils.pkb b/source/core/ut_utils.pkb index a8528fa86..6bcfbfeff 100644 --- a/source/core/ut_utils.pkb +++ b/source/core/ut_utils.pkb @@ -28,8 +28,8 @@ create or replace package body ut_utils is * Constants use in postfix and infix transformations */ gc_operators constant ut_varchar2_list := ut_varchar2_list('|','&','!'); - gc_unary_operator constant ut_varchar2_list := ut_varchar2_list('!'); -- right side associative operator - gc_binary_operator constant ut_varchar2_list := ut_varchar2_list('|','&'); -- left side associative operator + gc_unary_operators constant ut_varchar2_list := ut_varchar2_list('!'); -- right side associative operator + gc_binary_operators constant ut_varchar2_list := ut_varchar2_list('|','&'); -- left side associative operator type t_precedence_table is table of number index by varchar2(1); g_precedence t_precedence_table; @@ -1027,9 +1027,9 @@ create or replace package body ut_utils is --Exuecute modified shunting algorithm WHILE (l_idx is not null) loop l_token := l_input_tokens(l_idx); - if (l_token member of gc_operators and l_token member of gc_binary_operator) then + if (l_token member of gc_operators and l_token member of gc_binary_operators) then if not(l_expect_operator) then - raise ex_invalid_tag_expression; + raise_application_error(gc_invalid_tag_expression, 'Invalid Tag expression'); end if; while l_operator_stack.top > 0 and (g_precedence(l_operator_stack.peek) > g_precedence(l_token)) loop l_rnp_tokens.extend; @@ -1038,23 +1038,23 @@ create or replace package body ut_utils is l_operator_stack.push(l_input_tokens(l_idx)); l_expect_operand := true; l_expect_operator:= false; - elsif (l_token member of gc_operators and l_token member of gc_unary_operator) then + elsif (l_token member of gc_operators and l_token member of gc_unary_operators) then if not(l_expect_operand) then - raise ex_invalid_tag_expression; + raise_application_error(gc_invalid_tag_expression, 'Invalid Tag expression'); end if; l_operator_stack.push(l_input_tokens(l_idx)); l_expect_operand := true; l_expect_operator:= false; elsif l_token = '(' then if not(l_expect_operand) then - raise ex_invalid_tag_expression; + raise_application_error(gc_invalid_tag_expression, 'Invalid Tag expression'); end if; l_operator_stack.push(l_input_tokens(l_idx)); l_expect_operand := true; l_expect_operator:= false; elsif l_token = ')' then if not(l_expect_operator) then - raise ex_invalid_tag_expression; + raise_application_error(gc_invalid_tag_expression, 'Invalid Tag expression'); end if; while l_operator_stack.peek <> '(' loop l_rnp_tokens.extend; @@ -1065,7 +1065,7 @@ create or replace package body ut_utils is l_expect_operator:= true; else if not(l_expect_operand) then - raise ex_invalid_tag_expression; + raise_application_error(gc_invalid_tag_expression, 'Invalid Tag expression'); end if; l_rnp_tokens.extend; l_rnp_tokens(l_rnp_tokens.last) :=l_token; @@ -1078,7 +1078,7 @@ create or replace package body ut_utils is while l_operator_stack.peek is not null loop if l_operator_stack.peek in ('(',')') then - raise ex_invalid_tag_expression; + raise_application_error(gc_invalid_tag_expression, 'Invalid Tag expression'); end if; l_rnp_tokens.extend; l_rnp_tokens(l_rnp_tokens.last):=l_operator_stack.pop; @@ -1087,37 +1087,6 @@ create or replace package body ut_utils is return l_rnp_tokens; end shunt_logical_expression; - function convert_postfix_to_infix(a_postfix_exp in ut_varchar2_list) - return varchar2 is - l_infix_stack ut_stack := ut_stack(); - l_right_side varchar2(32767); - l_left_side varchar2(32767); - l_infix_exp varchar2(32767); - l_idx pls_integer; - begin - l_idx := a_postfix_exp.first; - while (l_idx is not null) loop - --If token is operand but also single tag - if a_postfix_exp(l_idx) not member of gc_operators then --its operand - l_infix_stack.push(a_postfix_exp(l_idx)); - --If token is unary operator not - elsif a_postfix_exp(l_idx) member of gc_unary_operator then - l_right_side := l_infix_stack.pop; - l_infix_exp := '('||a_postfix_exp(l_idx)||l_right_side||')'; - l_infix_stack.push(l_infix_exp); - --If token is binary operator - elsif a_postfix_exp(l_idx) member of gc_binary_operator then - l_right_side := l_infix_stack.pop; - l_left_side := l_infix_stack.pop; - l_infix_exp := '('||l_left_side||a_postfix_exp(l_idx)||l_right_side||')'; - l_infix_stack.push(l_infix_exp); - end if; - l_idx := a_postfix_exp.next(l_idx); - end loop; - - return l_infix_stack.pop; - end convert_postfix_to_infix; - function conv_postfix_to_infix_sql(a_postfix_exp in ut_varchar2_list) return varchar2 is l_infix_stack ut_stack := ut_stack(); @@ -1136,12 +1105,12 @@ create or replace package body ut_utils is elsif a_postfix_exp(l_idx) not member of gc_operators then l_infix_stack.push(a_postfix_exp(l_idx)); --If token is unary operator not - elsif a_postfix_exp(l_idx) member of gc_unary_operator then + elsif a_postfix_exp(l_idx) member of gc_unary_operators then l_right_side := l_infix_stack.pop; l_infix_exp := a_postfix_exp(l_idx)||'('||l_right_side||')'; l_infix_stack.push(l_infix_exp); --If token is binary operator - elsif a_postfix_exp(l_idx) member of gc_binary_operator then + elsif a_postfix_exp(l_idx) member of gc_binary_operators then l_right_side := l_infix_stack.pop; l_left_side := l_infix_stack.pop; l_infix_exp := '('||l_left_side||a_postfix_exp(l_idx)||l_right_side||')'; diff --git a/source/core/ut_utils.pks b/source/core/ut_utils.pks index 67b09b2f9..a7f7752fb 100644 --- a/source/core/ut_utils.pks +++ b/source/core/ut_utils.pks @@ -489,11 +489,6 @@ create or replace package ut_utils authid definer is */ function shunt_logical_expression(a_tags in varchar2) return ut_varchar2_list; - /* - * Function that converts postfix notation into infix - */ - function convert_postfix_to_infix(a_postfix_exp in ut_varchar2_list) return varchar2; - /* * Function that converts postfix notation into infix and creating a string of sql filter * that checking a tags collections for tags according to posted logic. diff --git a/test/ut3_tester/core/test_ut_utils.pkb b/test/ut3_tester/core/test_ut_utils.pkb index 22733c4ce..8f3b57fd0 100644 --- a/test/ut3_tester/core/test_ut_utils.pkb +++ b/test/ut3_tester/core/test_ut_utils.pkb @@ -511,27 +511,6 @@ end; ut.expect(l_postfix_string).to_equal('a!b|'); end; - procedure test_conv_from_rpn_to_infix is - l_postfix_rpn ut3_develop.ut_varchar2_list; - l_infix_string varchar2(4000); - begin - l_postfix_rpn := ut3_develop.ut_varchar2_list('A'); - l_infix_string := ut3_develop.ut_utils.convert_postfix_to_infix(l_postfix_rpn); - ut.expect(l_infix_string).to_equal('A'); - - l_postfix_rpn := ut3_develop.ut_varchar2_list('A','B','|'); - l_infix_string := ut3_develop.ut_utils.convert_postfix_to_infix(l_postfix_rpn); - ut.expect(l_infix_string).to_equal('(A|B)'); - - l_postfix_rpn := ut3_develop.ut_varchar2_list('a','b','|','c','d','&','|'); - l_infix_string := ut3_develop.ut_utils.convert_postfix_to_infix(l_postfix_rpn); - ut.expect(l_infix_string).to_equal('((a|b)|(c&d))'); - - l_postfix_rpn := ut3_develop.ut_varchar2_list('a','b','!','|'); - l_infix_string := ut3_develop.ut_utils.convert_postfix_to_infix(l_postfix_rpn); - ut.expect(l_infix_string).to_equal('(a|(!b))'); - end; - procedure conv_from_rpn_to_sql_filter is l_postfix_rpn ut3_develop.ut_varchar2_list; l_infix_string varchar2(4000); diff --git a/test/ut3_tester/core/test_ut_utils.pks b/test/ut3_tester/core/test_ut_utils.pks index 1561e1136..a58082be0 100644 --- a/test/ut3_tester/core/test_ut_utils.pks +++ b/test/ut3_tester/core/test_ut_utils.pks @@ -160,9 +160,6 @@ create or replace package test_ut_utils is --%test( Test conversion of expression into Reverse Polish Notation) procedure test_conversion_to_rpn; - --%test( Test conversion of expression from Reverse Polish Notation into infix) - procedure test_conv_from_rpn_to_infix; - --%test( Test conversion of expression from Reverse Polish Notation into custom where filter for SQL) procedure conv_from_rpn_to_sql_filter; From 02a071cb43799b8b68aa12cddb8cba757e6654a2 Mon Sep 17 00:00:00 2001 From: Lukasz Wasylow Date: Thu, 13 Apr 2023 16:47:00 -0700 Subject: [PATCH 47/77] Separate tag logic. --- source/core/ut_suite_cache_manager.pkb | 143 +-------- source/core/ut_suite_cache_manager.pks | 9 +- source/core/ut_suite_manager.pkb | 15 +- source/core/ut_suite_tag_filter.pkb | 288 ++++++++++++++++++ source/core/ut_suite_tag_filter.pks | 48 +++ source/core/ut_utils.pkb | 140 --------- source/core/ut_utils.pks | 18 -- source/install.sql | 2 + test/install_ut3_tester_tests.sql | 2 + .../core/test_ut_suite_tag_filter.pkb | 42 +++ .../core/test_ut_suite_tag_filter.pks | 13 + test/ut3_tester/core/test_ut_utils.pkb | 39 --- test/ut3_tester/core/test_ut_utils.pks | 6 - 13 files changed, 423 insertions(+), 342 deletions(-) create mode 100644 source/core/ut_suite_tag_filter.pkb create mode 100644 source/core/ut_suite_tag_filter.pks create mode 100644 test/ut3_tester/core/test_ut_suite_tag_filter.pkb create mode 100644 test/ut3_tester/core/test_ut_suite_tag_filter.pks diff --git a/source/core/ut_suite_cache_manager.pkb b/source/core/ut_suite_cache_manager.pkb index 858069c8f..3bb679517 100644 --- a/source/core/ut_suite_cache_manager.pkb +++ b/source/core/ut_suite_cache_manager.pkb @@ -220,121 +220,7 @@ create or replace package body ut_suite_cache_manager is and c.name = nvl(upper(sp.procedure_name),c.name)))) where r_num =1; return l_suite_items; end; - - /* - To support a legact tag notation - , = OR - - = NOT - we will perform a replace of that characters into - new notation. - | = OR - & = AND - ! = NOT - */ - function replace_legacy_tag_notation(a_tags varchar2 - ) return varchar2 is - l_tags ut_varchar2_list := ut_utils.string_to_table(a_tags,','); - l_tags_include varchar2(4000); - l_tags_exclude varchar2(4000); - l_return_tag varchar2(4000); - begin - if instr(a_tags,',') > 0 or instr(a_tags,'-') > 0 then - - select '('||listagg( t.column_value,'|') - within group( order by column_value)||')' - into l_tags_include - from table(l_tags) t - where t.column_value not like '-%'; - select '('||listagg( replace(t.column_value,'-','!'),' & ') - within group( order by column_value)||')' - into l_tags_exclude - from table(l_tags) t - where t.column_value like '-%'; - - - l_return_tag:= - case - when l_tags_include <> '()' and l_tags_exclude <> '()' - then l_tags_include || ' & ' || l_tags_exclude - when l_tags_include <> '()' - then l_tags_include - when l_tags_exclude <> '()' - then l_tags_exclude - end; - else - l_return_tag := a_tags; - end if; - return l_return_tag; - end; - - function create_where_filter(a_tags varchar2 - ) return varchar2 is - l_tags varchar2(4000); - begin - l_tags := replace(replace_legacy_tag_notation(a_tags),' '); - l_tags := ut_utils.conv_postfix_to_infix_sql(ut_utils.shunt_logical_expression(l_tags)); - l_tags := replace(l_tags, '|',' or '); - l_tags := replace(l_tags ,'&',' and '); - l_tags := replace(l_tags ,'!','not'); - return l_tags; - end; - - /* - Having a base set of suites we will do a further filter down if there are - any tags defined. - */ - function get_tags_suites ( - a_suite_items ut_suite_cache_rows, - a_tags varchar2 - ) return ut_suite_cache_rows is - l_suite_tags ut_suite_cache_rows := ut_suite_cache_rows(); - l_sql varchar2(32000); - l_tags varchar2(4000):= create_where_filter(a_tags); - begin - l_sql := - q'[ -with - suites_mv as ( - select c.id,value(c) as obj,c.path as path,c.self_type,c.object_owner,c.tags - from table(:suite_items) c - ), - suites_matching_expr as ( - select c.id,c.path as path,c.self_type,c.object_owner,c.tags - from suites_mv c - where c.self_type in ('UT_SUITE','UT_CONTEXT') - and ]'||l_tags||q'[ - ), - tests_matching_expr as ( - select c.id,c.path as path,c.self_type,c.object_owner,c.tags - from suites_mv c where c.self_type in ('UT_TEST') - and ]'||l_tags||q'[ - ), - tests_with_tags_inh_from_suite as ( - select c.id,c.self_type,c.path,c.tags multiset union distinct t.tags tags,c.object_owner - from suites_mv c join suites_matching_expr t - on (c.path||'.' like t.path || '.%' /*all descendants and self*/ and c.object_owner = t.object_owner) - ), - tests_with_tags_prom_to_suite as ( - select c.id,c.self_type,c.path,c.tags multiset union distinct t.tags tags,c.object_owner - from suites_mv c join tests_matching_expr t - on (t.path||'.' like c.path || '.%' /*all ancestors and self*/ and c.object_owner = t.object_owner) - ) - select obj from suites_mv c, - (select id,row_number() over (partition by id order by id) r_num from - (select id - from tests_with_tags_prom_to_suite tst - where ]'||l_tags||q'[ - union all - select id from tests_with_tags_inh_from_suite tst - where ]'||l_tags||q'[ - ) - ) t where c.id = t.id and r_num = 1 ]'; - - execute immediate l_sql bulk collect into l_suite_tags using a_suite_items; - return l_suite_tags; - end; - /* We will sort a suites in hierarchical structure. Sorting from bottom to top so when we consolidate @@ -387,30 +273,29 @@ with end; function get_cached_suite_rows( - a_schema_paths ut_path_items, - a_random_seed positive := null, - a_tags varchar2 := null + a_suites_filtered ut_suite_cache_rows ) return ut_suite_cache_rows is l_results ut_suite_cache_rows := ut_suite_cache_rows(); - l_suite_items ut_suite_cache_rows := ut_suite_cache_rows(); - l_schema_paths ut_path_items; - l_tags varchar2(4000) := a_tags; begin - l_schema_paths := a_schema_paths; - l_suite_items := get_suite_items(a_schema_paths); - if length(l_tags) > 0 then - l_suite_items := get_tags_suites(l_suite_items,l_tags); - end if; - - open c_get_bulk_cache_suite(l_suite_items); + open c_get_bulk_cache_suite(a_suites_filtered); fetch c_get_bulk_cache_suite bulk collect into l_results; close c_get_bulk_cache_suite; return l_results; end; - + function get_cached_suites( + a_schema_paths ut_path_items, + a_random_seed positive := null + ) return ut_suite_cache_rows is + l_suite_items ut_suite_cache_rows := ut_suite_cache_rows(); + l_schema_paths ut_path_items; + begin + l_schema_paths := a_schema_paths; + l_suite_items := get_suite_items(a_schema_paths); + return l_suite_items; + end; function get_schema_parse_time(a_schema_name varchar2) return timestamp result_cache is l_cache_parse_time timestamp; @@ -558,7 +443,7 @@ with a_schema_paths ut_path_items ) return ut_suite_cache_rows is begin - return get_cached_suite_rows( a_schema_paths ); + return get_cached_suite_rows(get_cached_suites( a_schema_paths )); end; function get_suite_items_info( diff --git a/source/core/ut_suite_cache_manager.pks b/source/core/ut_suite_cache_manager.pks index 7f06e95eb..c217abeed 100644 --- a/source/core/ut_suite_cache_manager.pks +++ b/source/core/ut_suite_cache_manager.pks @@ -55,11 +55,14 @@ create or replace package ut_suite_cache_manager authid definer is * Not to be used publicly. Used internally for building suites at runtime. */ function get_cached_suite_rows( + a_suites_filtered ut_suite_cache_rows + ) return ut_suite_cache_rows; + + function get_cached_suites( a_schema_paths ut_path_items, - a_random_seed positive := null, - a_tags varchar2 := null + a_random_seed positive := null ) return ut_suite_cache_rows; - + function get_schema_paths(a_paths in ut_varchar2_list) return ut_path_items; /* diff --git a/source/core/ut_suite_manager.pkb b/source/core/ut_suite_manager.pkb index 0f8629ded..f8a2e002c 100644 --- a/source/core/ut_suite_manager.pkb +++ b/source/core/ut_suite_manager.pkb @@ -355,17 +355,18 @@ create or replace package body ut_suite_manager is a_tags varchar2 := null, a_skip_all_objects boolean := false ) return t_cached_suites_cursor is - l_unfiltered_rows ut_suite_cache_rows; - l_filtered_rows ut_suite_cache_rows; - l_result t_cached_suites_cursor; + l_unfiltered_rows ut_suite_cache_rows; + l_tag_filter_applied ut_suite_cache_rows; + l_filtered_rows ut_suite_cache_rows; + l_result t_cached_suites_cursor; begin - l_unfiltered_rows := ut_suite_cache_manager.get_cached_suite_rows( + l_unfiltered_rows := ut_suite_cache_manager.get_cached_suites( a_schema_paths, - a_random_seed, - a_tags + a_random_seed ); - l_filtered_rows := get_filtered_cursor(l_unfiltered_rows,a_skip_all_objects); + l_tag_filter_applied := ut_suite_tag_filter.apply(l_unfiltered_rows,a_tags); + l_filtered_rows := get_filtered_cursor(ut_suite_cache_manager.get_cached_suite_rows(l_tag_filter_applied),a_skip_all_objects); reconcile_paths_and_suites(a_schema_paths,l_filtered_rows); ut_suite_cache_manager.sort_and_randomize_tests(l_filtered_rows,a_random_seed); diff --git a/source/core/ut_suite_tag_filter.pkb b/source/core/ut_suite_tag_filter.pkb new file mode 100644 index 000000000..f1e055a27 --- /dev/null +++ b/source/core/ut_suite_tag_filter.pkb @@ -0,0 +1,288 @@ +create or replace package body ut_suite_tag_filter is + /* + utPLSQL - Version 3 + Copyright 2016 - 2023 utPLSQL Project + + Licensed under the Apache License, Version 2.0 (the "License"): + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + */ + + /** + * Constants use in postfix and infix transformations + */ + gc_operators constant ut_varchar2_list := ut_varchar2_list('|','&','!'); + gc_unary_operators constant ut_varchar2_list := ut_varchar2_list('!'); -- right side associative operator + gc_binary_operators constant ut_varchar2_list := ut_varchar2_list('|','&'); -- left side associative operator + + type t_precedence_table is table of number index by varchar2(1); + g_precedence t_precedence_table; + + function tokenize_tags_string(a_tags in varchar2) return ut_varchar2_list is + l_tags_tokens ut_varchar2_list := ut_varchar2_list(); + begin + --Tokenize a string into operators and tags + select regexp_substr(a_tags,'([^!()|&]+)|([!()|&])', 1, level) as string_parts + bulk collect into l_tags_tokens + from dual connect by regexp_substr (a_tags, '([^!()|&]+)|([!()|&])', 1, level) is not null; + + return l_tags_tokens; + end; + + /* + To support a legact tag notation + , = OR + - = NOT + we will perform a replace of that characters into + new notation. + | = OR + & = AND + ! = NOT + */ + function replace_legacy_tag_notation(a_tags varchar2 + ) return varchar2 is + l_tags ut_varchar2_list := ut_utils.string_to_table(a_tags,','); + l_tags_include varchar2(4000); + l_tags_exclude varchar2(4000); + l_return_tag varchar2(4000); + begin + if instr(a_tags,',') > 0 or instr(a_tags,'-') > 0 then + + select '('||listagg( t.column_value,'|') + within group( order by column_value)||')' + into l_tags_include + from table(l_tags) t + where t.column_value not like '-%'; + + select '('||listagg( replace(t.column_value,'-','!'),' & ') + within group( order by column_value)||')' + into l_tags_exclude + from table(l_tags) t + where t.column_value like '-%'; + + + l_return_tag:= + case + when l_tags_include <> '()' and l_tags_exclude <> '()' + then l_tags_include || ' & ' || l_tags_exclude + when l_tags_include <> '()' + then l_tags_include + when l_tags_exclude <> '()' + then l_tags_exclude + end; + else + l_return_tag := a_tags; + end if; + return l_return_tag; + end; + + /* + https://stackoverflow.com/questions/29634992/shunting-yard-validate-expression + */ + function shunt_logical_expression(a_tags in varchar2) return ut_varchar2_list is + l_operator_stack ut_stack := ut_stack(); + l_input_tokens ut_varchar2_list := tokenize_tags_string(a_tags); + l_rnp_tokens ut_varchar2_list := ut_varchar2_list(); + l_token varchar2(32767); + l_expect_operand boolean := true; + l_expect_operator boolean := false; + l_idx pls_integer; + begin + l_idx := l_input_tokens.first; + --Exuecute modified shunting algorithm + WHILE (l_idx is not null) loop + l_token := l_input_tokens(l_idx); + if (l_token member of gc_operators and l_token member of gc_binary_operators) then + if not(l_expect_operator) then + raise_application_error(ut_utils.gc_invalid_tag_expression, 'Invalid Tag expression'); + end if; + while l_operator_stack.top > 0 and (g_precedence(l_operator_stack.peek) > g_precedence(l_token)) loop + l_rnp_tokens.extend; + l_rnp_tokens(l_rnp_tokens.last) := l_operator_stack.pop; + end loop; + l_operator_stack.push(l_input_tokens(l_idx)); + l_expect_operand := true; + l_expect_operator:= false; + elsif (l_token member of gc_operators and l_token member of gc_unary_operators) then + if not(l_expect_operand) then + raise_application_error(ut_utils.gc_invalid_tag_expression, 'Invalid Tag expression'); + end if; + l_operator_stack.push(l_input_tokens(l_idx)); + l_expect_operand := true; + l_expect_operator:= false; + elsif l_token = '(' then + if not(l_expect_operand) then + raise_application_error(ut_utils.gc_invalid_tag_expression, 'Invalid Tag expression'); + end if; + l_operator_stack.push(l_input_tokens(l_idx)); + l_expect_operand := true; + l_expect_operator:= false; + elsif l_token = ')' then + if not(l_expect_operator) then + raise_application_error(ut_utils.gc_invalid_tag_expression, 'Invalid Tag expression'); + end if; + while l_operator_stack.peek <> '(' loop + l_rnp_tokens.extend; + l_rnp_tokens(l_rnp_tokens.last) := l_operator_stack.pop; + end loop; + l_operator_stack.pop; --Pop the open bracket and discard it + l_expect_operand := false; + l_expect_operator:= true; + else + if not(l_expect_operand) then + raise_application_error(ut_utils.gc_invalid_tag_expression, 'Invalid Tag expression'); + end if; + l_rnp_tokens.extend; + l_rnp_tokens(l_rnp_tokens.last) :=l_token; + l_expect_operator := true; + l_expect_operand := false; + end if; + + l_idx := l_input_tokens.next(l_idx); + end loop; + + while l_operator_stack.peek is not null loop + if l_operator_stack.peek in ('(',')') then + raise_application_error(ut_utils.gc_invalid_tag_expression, 'Invalid Tag expression'); + end if; + l_rnp_tokens.extend; + l_rnp_tokens(l_rnp_tokens.last):=l_operator_stack.pop; + end loop; + + return l_rnp_tokens; + end shunt_logical_expression; + + function conv_postfix_to_infix_sql(a_postfix_exp in ut_varchar2_list) + return varchar2 is + l_infix_stack ut_stack := ut_stack(); + l_right_side varchar2(32767); + l_left_side varchar2(32767); + l_infix_exp varchar2(32767); + l_member_token varchar2(20) := ' member of tags'; + l_idx pls_integer; + begin + l_idx := a_postfix_exp.first; + while ( l_idx is not null) loop + --If token is operand but also single tag + if regexp_count(a_postfix_exp(l_idx),'[!()|&]') = 0 then + l_infix_stack.push(q'[']'||a_postfix_exp(l_idx)||q'[']'||l_member_token); + --If token is operand but containing other expressions + elsif a_postfix_exp(l_idx) not member of gc_operators then + l_infix_stack.push(a_postfix_exp(l_idx)); + --If token is unary operator not + elsif a_postfix_exp(l_idx) member of gc_unary_operators then + l_right_side := l_infix_stack.pop; + l_infix_exp := a_postfix_exp(l_idx)||'('||l_right_side||')'; + l_infix_stack.push(l_infix_exp); + --If token is binary operator + elsif a_postfix_exp(l_idx) member of gc_binary_operators then + l_right_side := l_infix_stack.pop; + l_left_side := l_infix_stack.pop; + l_infix_exp := '('||l_left_side||a_postfix_exp(l_idx)||l_right_side||')'; + l_infix_stack.push(l_infix_exp); + end if; + l_idx := a_postfix_exp.next(l_idx); + end loop; + + return l_infix_stack.pop; + end conv_postfix_to_infix_sql; + + function create_where_filter(a_tags varchar2 + ) return varchar2 is + l_tags varchar2(4000); + begin + l_tags := replace(replace_legacy_tag_notation(a_tags),' '); + l_tags := conv_postfix_to_infix_sql(shunt_logical_expression(l_tags)); + l_tags := replace(l_tags, '|',' or '); + l_tags := replace(l_tags ,'&',' and '); + l_tags := replace(l_tags ,'!','not'); + return l_tags; + end; + + + /* + Having a base set of suites we will do a further filter down if there are + any tags defined. + */ + function get_tags_suites ( + a_suite_items ut_suite_cache_rows, + a_tags varchar2 + ) return ut_suite_cache_rows is + l_suite_tags ut_suite_cache_rows := ut_suite_cache_rows(); + l_sql varchar2(32000); + l_tags varchar2(4000):= create_where_filter(a_tags); + begin + l_sql := + q'[ +with + suites_mv as ( + select c.id,value(c) as obj,c.path as path,c.self_type,c.object_owner,c.tags + from table(:suite_items) c + ), + suites_matching_expr as ( + select c.id,c.path as path,c.self_type,c.object_owner,c.tags + from suites_mv c + where c.self_type in ('UT_SUITE','UT_CONTEXT') + and ]'||l_tags||q'[ + ), + tests_matching_expr as ( + select c.id,c.path as path,c.self_type,c.object_owner,c.tags + from suites_mv c where c.self_type in ('UT_TEST') + and ]'||l_tags||q'[ + ), + tests_with_tags_inh_from_suite as ( + select c.id,c.self_type,c.path,c.tags multiset union distinct t.tags tags,c.object_owner + from suites_mv c join suites_matching_expr t + on (c.path||'.' like t.path || '.%' /*all descendants and self*/ and c.object_owner = t.object_owner) + ), + tests_with_tags_prom_to_suite as ( + select c.id,c.self_type,c.path,c.tags multiset union distinct t.tags tags,c.object_owner + from suites_mv c join tests_matching_expr t + on (t.path||'.' like c.path || '.%' /*all ancestors and self*/ and c.object_owner = t.object_owner) + ) + select obj from suites_mv c, + (select id,row_number() over (partition by id order by id) r_num from + (select id + from tests_with_tags_prom_to_suite tst + where ]'||l_tags||q'[ + union all + select id from tests_with_tags_inh_from_suite tst + where ]'||l_tags||q'[ + ) + ) t where c.id = t.id and r_num = 1 ]'; + + execute immediate l_sql bulk collect into l_suite_tags using a_suite_items; + return l_suite_tags; + end; + + function apply( + a_unfiltered_rows ut_suite_cache_rows, + a_tags varchar2 := null + ) return ut_suite_cache_rows is + l_suite_items ut_suite_cache_rows := a_unfiltered_rows; + begin + if length(a_tags) > 0 then + l_suite_items := get_tags_suites(l_suite_items,a_tags); + end if; + + return l_suite_items; + end; + +begin + --Define operators precedence + g_precedence('!'):=4; + g_precedence('&'):=3; + g_precedence('|'):=2; + g_precedence(')'):=1; + g_precedence('('):=1; + +end ut_suite_tag_filter; +/ diff --git a/source/core/ut_suite_tag_filter.pks b/source/core/ut_suite_tag_filter.pks new file mode 100644 index 000000000..787c72603 --- /dev/null +++ b/source/core/ut_suite_tag_filter.pks @@ -0,0 +1,48 @@ +create or replace package ut_suite_tag_filter authid definer is + /* + utPLSQL - Version 3 + Copyright 2016 - 2023 utPLSQL Project + + Licensed under the Apache License, Version 2.0 (the "License"): + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. + */ + + /** + * Package that will filter suites by tags + * + */ + + /* + * Return table of tokens character by character + */ + function tokenize_tags_string(a_tags in varchar2) return ut_varchar2_list; + + /* + * Function that uses Dijkstra algorithm to parse mathematical and logical expression + * and return a list of elements in Reverse Polish Notation ( postfix ) + * As part of execution it will validate expression. + */ + function shunt_logical_expression(a_tags in varchar2) return ut_varchar2_list; + + /* + * Function that converts postfix notation into infix and creating a string of sql filter + * that checking a tags collections for tags according to posted logic. + */ + function conv_postfix_to_infix_sql(a_postfix_exp in ut_varchar2_list) return varchar2; + + function apply( + a_unfiltered_rows ut_suite_cache_rows, + a_tags varchar2 := null + ) return ut_suite_cache_rows; + +end ut_suite_tag_filter; +/ diff --git a/source/core/ut_utils.pkb b/source/core/ut_utils.pkb index 6bcfbfeff..6f972e11c 100644 --- a/source/core/ut_utils.pkb +++ b/source/core/ut_utils.pkb @@ -24,15 +24,6 @@ create or replace package body ut_utils is gc_full_valid_xml_name constant varchar2(50) := '^([_a-zA-Z])([_a-zA-Z0-9\.-])*$'; gc_owner_hash constant integer(11) := dbms_utility.get_hash_value( ut_owner(), 0, power(2,31)-1); - /** - * Constants use in postfix and infix transformations - */ - gc_operators constant ut_varchar2_list := ut_varchar2_list('|','&','!'); - gc_unary_operators constant ut_varchar2_list := ut_varchar2_list('!'); -- right side associative operator - gc_binary_operators constant ut_varchar2_list := ut_varchar2_list('|','&'); -- left side associative operator - - type t_precedence_table is table of number index by varchar2(1); - g_precedence t_precedence_table; function surround_with(a_value varchar2, a_quote_char varchar2) return varchar2 is begin @@ -999,136 +990,5 @@ create or replace package body ut_utils is return l_result; end; - - function tokenize_tags_string(a_tags in varchar2) return ut_varchar2_list is - l_tags_tokens ut_varchar2_list := ut_varchar2_list(); - begin - --Tokenize a string into operators and tags - select regexp_substr(a_tags,'([^!()|&]+)|([!()|&])', 1, level) as string_parts - bulk collect into l_tags_tokens - from dual connect by regexp_substr (a_tags, '([^!()|&]+)|([!()|&])', 1, level) is not null; - - return l_tags_tokens; - end; - - /* - https://stackoverflow.com/questions/29634992/shunting-yard-validate-expression - */ - function shunt_logical_expression(a_tags in varchar2) return ut_varchar2_list is - l_operator_stack ut_stack := ut_stack(); - l_input_tokens ut_varchar2_list := tokenize_tags_string(a_tags); - l_rnp_tokens ut_varchar2_list := ut_varchar2_list(); - l_token varchar2(32767); - l_expect_operand boolean := true; - l_expect_operator boolean := false; - l_idx pls_integer; - begin - l_idx := l_input_tokens.first; - --Exuecute modified shunting algorithm - WHILE (l_idx is not null) loop - l_token := l_input_tokens(l_idx); - if (l_token member of gc_operators and l_token member of gc_binary_operators) then - if not(l_expect_operator) then - raise_application_error(gc_invalid_tag_expression, 'Invalid Tag expression'); - end if; - while l_operator_stack.top > 0 and (g_precedence(l_operator_stack.peek) > g_precedence(l_token)) loop - l_rnp_tokens.extend; - l_rnp_tokens(l_rnp_tokens.last) := l_operator_stack.pop; - end loop; - l_operator_stack.push(l_input_tokens(l_idx)); - l_expect_operand := true; - l_expect_operator:= false; - elsif (l_token member of gc_operators and l_token member of gc_unary_operators) then - if not(l_expect_operand) then - raise_application_error(gc_invalid_tag_expression, 'Invalid Tag expression'); - end if; - l_operator_stack.push(l_input_tokens(l_idx)); - l_expect_operand := true; - l_expect_operator:= false; - elsif l_token = '(' then - if not(l_expect_operand) then - raise_application_error(gc_invalid_tag_expression, 'Invalid Tag expression'); - end if; - l_operator_stack.push(l_input_tokens(l_idx)); - l_expect_operand := true; - l_expect_operator:= false; - elsif l_token = ')' then - if not(l_expect_operator) then - raise_application_error(gc_invalid_tag_expression, 'Invalid Tag expression'); - end if; - while l_operator_stack.peek <> '(' loop - l_rnp_tokens.extend; - l_rnp_tokens(l_rnp_tokens.last) := l_operator_stack.pop; - end loop; - l_operator_stack.pop; --Pop the open bracket and discard it - l_expect_operand := false; - l_expect_operator:= true; - else - if not(l_expect_operand) then - raise_application_error(gc_invalid_tag_expression, 'Invalid Tag expression'); - end if; - l_rnp_tokens.extend; - l_rnp_tokens(l_rnp_tokens.last) :=l_token; - l_expect_operator := true; - l_expect_operand := false; - end if; - - l_idx := l_input_tokens.next(l_idx); - end loop; - - while l_operator_stack.peek is not null loop - if l_operator_stack.peek in ('(',')') then - raise_application_error(gc_invalid_tag_expression, 'Invalid Tag expression'); - end if; - l_rnp_tokens.extend; - l_rnp_tokens(l_rnp_tokens.last):=l_operator_stack.pop; - end loop; - - return l_rnp_tokens; - end shunt_logical_expression; - - function conv_postfix_to_infix_sql(a_postfix_exp in ut_varchar2_list) - return varchar2 is - l_infix_stack ut_stack := ut_stack(); - l_right_side varchar2(32767); - l_left_side varchar2(32767); - l_infix_exp varchar2(32767); - l_member_token varchar2(20) := ' member of tags'; - l_idx pls_integer; - begin - l_idx := a_postfix_exp.first; - while ( l_idx is not null) loop - --If token is operand but also single tag - if regexp_count(a_postfix_exp(l_idx),'[!()|&]') = 0 then - l_infix_stack.push(q'[']'||a_postfix_exp(l_idx)||q'[']'||l_member_token); - --If token is operand but containing other expressions - elsif a_postfix_exp(l_idx) not member of gc_operators then - l_infix_stack.push(a_postfix_exp(l_idx)); - --If token is unary operator not - elsif a_postfix_exp(l_idx) member of gc_unary_operators then - l_right_side := l_infix_stack.pop; - l_infix_exp := a_postfix_exp(l_idx)||'('||l_right_side||')'; - l_infix_stack.push(l_infix_exp); - --If token is binary operator - elsif a_postfix_exp(l_idx) member of gc_binary_operators then - l_right_side := l_infix_stack.pop; - l_left_side := l_infix_stack.pop; - l_infix_exp := '('||l_left_side||a_postfix_exp(l_idx)||l_right_side||')'; - l_infix_stack.push(l_infix_exp); - end if; - l_idx := a_postfix_exp.next(l_idx); - end loop; - - return l_infix_stack.pop; - end conv_postfix_to_infix_sql; - -begin - --Define operator precedence - g_precedence('!'):=4; - g_precedence('&'):=3; - g_precedence('|'):=2; - g_precedence(')'):=1; - g_precedence('('):=1; - end ut_utils; / diff --git a/source/core/ut_utils.pks b/source/core/ut_utils.pks index a7f7752fb..b60fc4cdf 100644 --- a/source/core/ut_utils.pks +++ b/source/core/ut_utils.pks @@ -477,23 +477,5 @@ create or replace package ut_utils authid definer is */ function interval_to_text(a_interval yminterval_unconstrained) return varchar2; - /* - * Return table of tokens character by character - */ - function tokenize_tags_string(a_tags in varchar2) return ut_varchar2_list; - - /* - * Function that uses Dijkstra algorithm to parse mathematical and logical expression - * and return a list of elements in Reverse Polish Notation ( postfix ) - * As part of execution it will validate expression. - */ - function shunt_logical_expression(a_tags in varchar2) return ut_varchar2_list; - - /* - * Function that converts postfix notation into infix and creating a string of sql filter - * that checking a tags collections for tags according to posted logic. - */ - function conv_postfix_to_infix_sql(a_postfix_exp in ut_varchar2_list) return varchar2; - end ut_utils; / diff --git a/source/install.sql b/source/install.sql index a54977f94..827213e6c 100644 --- a/source/install.sql +++ b/source/install.sql @@ -155,6 +155,8 @@ create or replace context &&ut3_owner._info using &&ut3_owner..ut_session_contex @@install_component.sql 'core/ut_suite_cache_seq.sql' @@install_component.sql 'core/ut_suite_cache.sql' +@@install_component.sql 'core/ut_suite_tag_filter.pks' +@@install_component.sql 'core/ut_suite_tag_filter.pkb' @@install_component.sql 'core/ut_suite_cache_manager.pks' @@install_component.sql 'core/ut_suite_cache_manager.pkb' @@install_component.sql 'core/ut_suite_builder.pks' diff --git a/test/install_ut3_tester_tests.sql b/test/install_ut3_tester_tests.sql index 8163a39f2..cc96b2b64 100644 --- a/test/install_ut3_tester_tests.sql +++ b/test/install_ut3_tester_tests.sql @@ -17,6 +17,7 @@ alter session set plsql_optimize_level=0; @@ut3_tester/core/annotations/test_annot_disabled_reason.pks @@ut3_tester/core/expectations/test_expectation_processor.pks @@ut3_tester/core/test_ut_utils.pks +@@ut3_tester/core/test_ut_suite_tag_filter.pks @@ut3_tester/core/test_ut_test.pks @@ut3_tester/core/test_ut_suite.pks @@ut3_tester/core/test_ut_executable.pks @@ -35,6 +36,7 @@ alter session set plsql_optimize_level=0; @@ut3_tester/core/annotations/test_annot_disabled_reason.pkb @@ut3_tester/core/expectations/test_expectation_processor.pkb @@ut3_tester/core/test_ut_utils.pkb +@@ut3_tester/core/test_ut_suite_tag_filter.pkb @@ut3_tester/core/test_ut_test.pkb @@ut3_tester/core/test_ut_suite.pkb @@ut3_tester/core/test_ut_executable.pkb diff --git a/test/ut3_tester/core/test_ut_suite_tag_filter.pkb b/test/ut3_tester/core/test_ut_suite_tag_filter.pkb new file mode 100644 index 000000000..a1d0be1f7 --- /dev/null +++ b/test/ut3_tester/core/test_ut_suite_tag_filter.pkb @@ -0,0 +1,42 @@ +create or replace package body test_ut_suite_tag_filter is + + procedure test_conversion_to_rpn is + l_postfix ut3_develop.ut_varchar2_list; + l_postfix_string varchar2(4000); + begin + l_postfix := ut3_develop.ut_suite_tag_filter.shunt_logical_expression('A'); + l_postfix_string := ut3_develop.ut_utils.table_to_clob(l_postfix,''); + ut.expect(l_postfix_string).to_equal('A'); + + l_postfix := ut3_develop.ut_suite_tag_filter.shunt_logical_expression('A|B'); + l_postfix_string := ut3_develop.ut_utils.table_to_clob(l_postfix,''); + ut.expect(l_postfix_string).to_equal('AB|'); + + l_postfix := ut3_develop.ut_suite_tag_filter.shunt_logical_expression('(a|b)|c&d'); + l_postfix_string := ut3_develop.ut_utils.table_to_clob(l_postfix,''); + ut.expect(l_postfix_string).to_equal('ab|cd&|'); + + l_postfix := ut3_develop.ut_suite_tag_filter.shunt_logical_expression('!a|b'); + l_postfix_string := ut3_develop.ut_utils.table_to_clob(l_postfix,''); + ut.expect(l_postfix_string).to_equal('a!b|'); + end; + + procedure conv_from_rpn_to_sql_filter is + l_postfix_rpn ut3_develop.ut_varchar2_list; + l_infix_string varchar2(4000); + begin + l_postfix_rpn := ut3_develop.ut_varchar2_list('A'); + l_infix_string := ut3_develop.ut_suite_tag_filter.conv_postfix_to_infix_sql(l_postfix_rpn); + ut.expect(l_infix_string).to_equal(q'['A' member of tags]'); + + l_postfix_rpn := ut3_develop.ut_varchar2_list('A','B','|'); + l_infix_string := ut3_develop.ut_suite_tag_filter.conv_postfix_to_infix_sql(l_postfix_rpn); + ut.expect(l_infix_string).to_equal(q'[('A' member of tags|'B' member of tags)]'); + + l_postfix_rpn := ut3_develop.ut_varchar2_list('a','b','!','|'); + l_infix_string := ut3_develop.ut_suite_tag_filter.conv_postfix_to_infix_sql(l_postfix_rpn); + ut.expect(l_infix_string).to_equal(q'[('a' member of tags|!('b' member of tags))]'); + end; + +end test_ut_suite_tag_filter; +/ diff --git a/test/ut3_tester/core/test_ut_suite_tag_filter.pks b/test/ut3_tester/core/test_ut_suite_tag_filter.pks new file mode 100644 index 000000000..093fab856 --- /dev/null +++ b/test/ut3_tester/core/test_ut_suite_tag_filter.pks @@ -0,0 +1,13 @@ +create or replace package test_ut_suite_tag_filter is + + --%suite(ut_suite_tag_filter) + --%suitepath(utplsql.ut3_tester.core) + + --%test( Test conversion of expression into Reverse Polish Notation) + procedure test_conversion_to_rpn; + + --%test( Test conversion of expression from Reverse Polish Notation into custom where filter for SQL) + procedure conv_from_rpn_to_sql_filter; + +end test_ut_suite_tag_filter; +/ diff --git a/test/ut3_tester/core/test_ut_utils.pkb b/test/ut3_tester/core/test_ut_utils.pkb index 8f3b57fd0..ec7e4f403 100644 --- a/test/ut3_tester/core/test_ut_utils.pkb +++ b/test/ut3_tester/core/test_ut_utils.pkb @@ -489,44 +489,5 @@ end; ut.expect(l_expected).to_equal(l_actual); end; - - procedure test_conversion_to_rpn is - l_postfix ut3_develop.ut_varchar2_list; - l_postfix_string varchar2(4000); - begin - l_postfix := ut3_develop.ut_utils.shunt_logical_expression('A'); - l_postfix_string := ut3_develop.ut_utils.table_to_clob(l_postfix,''); - ut.expect(l_postfix_string).to_equal('A'); - - l_postfix := ut3_develop.ut_utils.shunt_logical_expression('A|B'); - l_postfix_string := ut3_develop.ut_utils.table_to_clob(l_postfix,''); - ut.expect(l_postfix_string).to_equal('AB|'); - - l_postfix := ut3_develop.ut_utils.shunt_logical_expression('(a|b)|c&d'); - l_postfix_string := ut3_develop.ut_utils.table_to_clob(l_postfix,''); - ut.expect(l_postfix_string).to_equal('ab|cd&|'); - - l_postfix := ut3_develop.ut_utils.shunt_logical_expression('!a|b'); - l_postfix_string := ut3_develop.ut_utils.table_to_clob(l_postfix,''); - ut.expect(l_postfix_string).to_equal('a!b|'); - end; - - procedure conv_from_rpn_to_sql_filter is - l_postfix_rpn ut3_develop.ut_varchar2_list; - l_infix_string varchar2(4000); - begin - l_postfix_rpn := ut3_develop.ut_varchar2_list('A'); - l_infix_string := ut3_develop.ut_utils.conv_postfix_to_infix_sql(l_postfix_rpn); - ut.expect(l_infix_string).to_equal(q'['A' member of tags]'); - - l_postfix_rpn := ut3_develop.ut_varchar2_list('A','B','|'); - l_infix_string := ut3_develop.ut_utils.conv_postfix_to_infix_sql(l_postfix_rpn); - ut.expect(l_infix_string).to_equal(q'[('A' member of tags|'B' member of tags)]'); - - l_postfix_rpn := ut3_develop.ut_varchar2_list('a','b','!','|'); - l_infix_string := ut3_develop.ut_utils.conv_postfix_to_infix_sql(l_postfix_rpn); - ut.expect(l_infix_string).to_equal(q'[('a' member of tags|!('b' member of tags))]'); - end; - end test_ut_utils; / diff --git a/test/ut3_tester/core/test_ut_utils.pks b/test/ut3_tester/core/test_ut_utils.pks index a58082be0..114b35e86 100644 --- a/test/ut3_tester/core/test_ut_utils.pks +++ b/test/ut3_tester/core/test_ut_utils.pks @@ -157,11 +157,5 @@ create or replace package test_ut_utils is --%endcontext - --%test( Test conversion of expression into Reverse Polish Notation) - procedure test_conversion_to_rpn; - - --%test( Test conversion of expression from Reverse Polish Notation into custom where filter for SQL) - procedure conv_from_rpn_to_sql_filter; - end test_ut_utils; / From b8b66ee715c6765d5fef1ad92d8c25513bf757c5 Mon Sep 17 00:00:00 2001 From: Lukasz Wasylow Date: Thu, 13 Apr 2023 17:00:41 -0700 Subject: [PATCH 48/77] Fix uninstall --- source/uninstall_objects.sql | 2 ++ 1 file changed, 2 insertions(+) diff --git a/source/uninstall_objects.sql b/source/uninstall_objects.sql index 1e21f0f95..f488b8b8d 100644 --- a/source/uninstall_objects.sql +++ b/source/uninstall_objects.sql @@ -95,6 +95,8 @@ drop package ut_suite_manager; drop package ut_suite_builder; +drop package ut_suite_tag_filter; + drop package ut_suite_cache_manager; drop table ut_suite_cache purge; From 077fdb11316cda8b99b9ccc909a1c9cb3de57267 Mon Sep 17 00:00:00 2001 From: Lukasz Wasylow Date: Fri, 14 Apr 2023 15:48:56 -0700 Subject: [PATCH 49/77] Various PR fixe --- docs/userguide/annotations.md | 82 +------------------ docs/userguide/running-unit-tests.md | 76 +++++++++++++++-- source/core/ut_suite_tag_filter.pkb | 18 ++-- source/core/ut_suite_tag_filter.pks | 2 +- .../core/test_ut_suite_tag_filter.pkb | 49 ++++++++++- .../core/test_ut_suite_tag_filter.pks | 20 +++++ 6 files changed, 145 insertions(+), 102 deletions(-) diff --git a/docs/userguide/annotations.md b/docs/userguide/annotations.md index 8e5a5d1ca..552b02b52 100644 --- a/docs/userguide/annotations.md +++ b/docs/userguide/annotations.md @@ -1616,7 +1616,7 @@ or Tags are defined as a comma separated list within the `--%tags` annotation. -When a suite/context is tagged, all of its children will automatically inherit the tag and get executed along with the parent, unless they are excluded by tag expression. +When a suite/context is tagged, all of its children will automatically inherit the tag and get executed along with the parent, unless they are excluded explicitly at runtime with a negated tag expression. Parent suite tests are not executed, but a suitepath hierarchy is kept. Sample test suite package with tags. @@ -1657,52 +1657,6 @@ create or replace package body ut_sample_test is end ut_sample_test; / ``` - -#### Tag Expressions - -Tag expressions are boolean expressions with the operators !, & and |. In addition, ( and ) can be used to adjust for operator precedence. - -| Operator | Meaning | -| -------- | --------| -| ! | not | -| & | and | -| \| | or | - -If you are tagging your tests across multiple dimensions, tag expressions help you to select which tests to execute. When tagging by test type (e.g., micro, integration, end-to-end) and feature (e.g., product, catalog, shipping), the following tag expressions can be useful. - - -| Tag Expression | Selection | -| -------- | --------| -| product | all tests for product | -| catalog \| shipping | all tests for catalog plus all tests for shipping | -| catalog & shipping | all tests for the intersection between catalog and shipping | -| product & !end-to-end | all tests for product, but not the end-to-end tests | -| (micro \| integration) & (product \| shipping) | all micro or integration tests for product or shipping | - - -Execution of the test is done by using the parameter `a_tags` with tag expressions - - -```sql linenums="1" -select * from table(ut.run(a_tags => 'fast|!complex')); -``` -The above call will execute all tests from `ut_sample_test` package as the whole suite is tagged with `api` because a suite meet expression condition. - -```sql linenums="1" -select * from table(ut.run(a_path => 'ut_sample_test',a_tags => 'api')); -``` -The above call will execute all tests from `ut_sample_test` package as the whole suite is tagged with `api` - -```sql linenums="1" -select * from table(ut.run(a_tags => 'complex')); -``` -The above call will execute only the `ut_sample_test.ut_refcursors1` test, as only the test `ut_refcursors1` is tagged with `complex` - -```sql linenums="1" -select * from table(ut.run(a_tags => 'fast')); -``` -The above call will execute both `ut_sample_test.ut_refcursors1` and `ut_sample_test.ut_test` tests, as both tests are tagged with `fast` - #### Tag naming convention Tags must follow the below naming convention: @@ -1715,45 +1669,11 @@ Tags must follow the below naming convention: - vertical bar (|) - exclamation point (!) - tag cannot be null or blank -- tag cannot contain whitespace - tag cannot start with a dash, e.g. `-some-stuff` is **not** a valid tag - tag cannot contain spaces, e.g. `test of batch`. To create a multi-word tag use underscores or dashes, e.g. `test_of_batch`, `test-of-batch` - leading and trailing spaces are ignored in tag name, e.g. `--%tags( tag1 , tag2 )` becomes `tag1` and `tag2` tag names -#### Excluding tests/suites by tags - -It is possible to exclude parts of test suites with tags. -In order to do so, prefix the tag name to exclude with a `!` (exclamation) sign when invoking the test run which is equivalent of `-` (dash) in legacy notation. -Examples (based on above sample test suite) - -```sql linenums="1" -select * from table(ut.run(a_tags => '(api|fast)&!complex')); -``` - -which is equivalent of legacy calling: - -```sql linenums="1" -select * from table(ut.run(a_tags => 'api,fast,-complex')); -``` - -or - -```sql linenums="1" -select * from table(ut.run(a_tags => '(api|fast)&(!complex&!test1)')); -``` - -which is equivalent of legacy calling: - -```sql linenums="1" -select * from table(ut.run(a_tags => 'api,fast,-complex,-test1')); -``` - -The above call will execute all suites/contexts/tests that are marked with any of tags `api` or `fast` except those suites/contexts/tests that are marked as `complex`. -Given the above example package `ut_sample_test`, only `ut_sample_test.ut_test` will be executed. - - - ### Suitepath It is very likely that the application for which you are going to introduce tests consists of many different packages, procedures and functions. diff --git a/docs/userguide/running-unit-tests.md b/docs/userguide/running-unit-tests.md index 978803b9a..9abc7c964 100644 --- a/docs/userguide/running-unit-tests.md +++ b/docs/userguide/running-unit-tests.md @@ -319,23 +319,85 @@ select * from table(ut.run('hr.test_apply_bonus', a_random_test_order_seed => 30 In addition to the path, you can filter the tests to be run by specifying tags. Tags are defined in the test / context / suite with the `--%tags`-annotation ([Read more](annotations.md#tags)). Multiple tags are separated by comma. -The framework applies `OR` logic to all specified tags so any test / suite that matches at least one tag will be included in the test run. + + +### Tag Expressions + +Tag expressions are boolean expressions with the operators !, & and |. In addition, ( and ) can be used to adjust for operator precedence. + +| Operator | Meaning | +| -------- | --------| +| ! | not | +| & | and | +| \| | or | + +If you are tagging your tests across multiple dimensions, tag expressions help you to select which tests to execute. When tagging by test type (e.g., micro, integration, end-to-end) and feature (e.g., product, catalog, shipping), the following tag expressions can be useful. + + +| Tag Expression | Selection | +| -------- | --------| +| product | all tests for product | +| catalog \| shipping | all tests for catalog plus all tests for shipping | +| catalog & shipping | all tests for the intersection between catalog and shipping | +| product & !end-to-end | all tests for product, but not the end-to-end tests | +| (micro \| integration) & (product \| shipping) | all micro or integration tests for product or shipping | + + +Execution of the test is done by using the parameter `a_tags` with tag expressions + ```sql linenums="1" -begin - ut.run('hr.test_apply_bonus', a_tags => 'test1,test2'); -end; +select * from table(ut.run(a_tags => 'fast|!complex')); +``` +The above call will execute all tests from `ut_sample_test` package as the whole suite is tagged with `api` because a suite meet expression condition. + +```sql linenums="1" +select * from table(ut.run(a_path => 'ut_sample_test',a_tags => 'api')); +``` +The above call will execute all tests from `ut_sample_test` package as the whole suite is tagged with `api` + +```sql linenums="1" +select * from table(ut.run(a_tags => 'complex')); ``` +The above call will execute only the `ut_sample_test.ut_refcursors1` test, as only the test `ut_refcursors1` is tagged with `complex` + ```sql linenums="1" -select * from table(ut.run('hr.test_apply_bonus', a_tags => 'suite1')) +select * from table(ut.run(a_tags => 'fast')); ``` +The above call will execute both `ut_sample_test.ut_refcursors1` and `ut_sample_test.ut_test` tests, as both tests are tagged with `fast` + +### Excluding tests/suites by tags -You can also exclude specific tags by adding a `-` (dash) in front of the tag +It is possible to exclude parts of test suites with tags. +In order to do so, prefix the tag name to exclude with a `!` (exclamation) sign when invoking the test run which is equivalent of `-` (dash) in legacy notation. +Examples (based on above sample test suite) ```sql linenums="1" -select * from table(ut.run('hr.test_apply_bonus', a_tags => '-suite1')) +select * from table(ut.run(a_tags => '(api|fast)&!complex')); ``` +which is equivalent of legacy calling: + +```sql linenums="1" +select * from table(ut.run(a_tags => 'api,fast,-complex')); +``` + +or + +```sql linenums="1" +select * from table(ut.run(a_tags => '(api|fast)&(!complex&!test1)')); +``` + +which is equivalent of legacy calling: + +```sql linenums="1" +select * from table(ut.run(a_tags => 'api,fast,-complex,-test1')); +``` + +The above call will execute all suites/contexts/tests that are marked with any of tags `api` or `fast` except those suites/contexts/tests that are marked as `complex`. +Given the above example package `ut_sample_test`, only `ut_sample_test.ut_test` will be executed. + + ## Keeping uncommitted data after test-run utPLSQL by default runs tests in autonomous transaction and performs automatic rollback to assure that tests do not impact one-another and do not have impact on the current session in your IDE. diff --git a/source/core/ut_suite_tag_filter.pkb b/source/core/ut_suite_tag_filter.pkb index f1e055a27..d5df54212 100644 --- a/source/core/ut_suite_tag_filter.pkb +++ b/source/core/ut_suite_tag_filter.pkb @@ -87,19 +87,18 @@ create or replace package body ut_suite_tag_filter is /* https://stackoverflow.com/questions/29634992/shunting-yard-validate-expression */ - function shunt_logical_expression(a_tags in varchar2) return ut_varchar2_list is + function shunt_logical_expression(a_tags in ut_varchar2_list) return ut_varchar2_list is l_operator_stack ut_stack := ut_stack(); - l_input_tokens ut_varchar2_list := tokenize_tags_string(a_tags); l_rnp_tokens ut_varchar2_list := ut_varchar2_list(); l_token varchar2(32767); l_expect_operand boolean := true; l_expect_operator boolean := false; l_idx pls_integer; begin - l_idx := l_input_tokens.first; + l_idx := a_tags.first; --Exuecute modified shunting algorithm WHILE (l_idx is not null) loop - l_token := l_input_tokens(l_idx); + l_token := a_tags(l_idx); if (l_token member of gc_operators and l_token member of gc_binary_operators) then if not(l_expect_operator) then raise_application_error(ut_utils.gc_invalid_tag_expression, 'Invalid Tag expression'); @@ -108,21 +107,21 @@ create or replace package body ut_suite_tag_filter is l_rnp_tokens.extend; l_rnp_tokens(l_rnp_tokens.last) := l_operator_stack.pop; end loop; - l_operator_stack.push(l_input_tokens(l_idx)); + l_operator_stack.push(a_tags(l_idx)); l_expect_operand := true; l_expect_operator:= false; elsif (l_token member of gc_operators and l_token member of gc_unary_operators) then if not(l_expect_operand) then raise_application_error(ut_utils.gc_invalid_tag_expression, 'Invalid Tag expression'); end if; - l_operator_stack.push(l_input_tokens(l_idx)); + l_operator_stack.push(a_tags(l_idx)); l_expect_operand := true; l_expect_operator:= false; elsif l_token = '(' then if not(l_expect_operand) then raise_application_error(ut_utils.gc_invalid_tag_expression, 'Invalid Tag expression'); end if; - l_operator_stack.push(l_input_tokens(l_idx)); + l_operator_stack.push(a_tags(l_idx)); l_expect_operand := true; l_expect_operator:= false; elsif l_token = ')' then @@ -146,7 +145,7 @@ create or replace package body ut_suite_tag_filter is l_expect_operand := false; end if; - l_idx := l_input_tokens.next(l_idx); + l_idx := a_tags.next(l_idx); end loop; while l_operator_stack.peek is not null loop @@ -198,9 +197,10 @@ create or replace package body ut_suite_tag_filter is function create_where_filter(a_tags varchar2 ) return varchar2 is l_tags varchar2(4000); + l_tokenized_tags ut_varchar2_list; begin l_tags := replace(replace_legacy_tag_notation(a_tags),' '); - l_tags := conv_postfix_to_infix_sql(shunt_logical_expression(l_tags)); + l_tags := conv_postfix_to_infix_sql(shunt_logical_expression(tokenize_tags_string(l_tags))); l_tags := replace(l_tags, '|',' or '); l_tags := replace(l_tags ,'&',' and '); l_tags := replace(l_tags ,'!','not'); diff --git a/source/core/ut_suite_tag_filter.pks b/source/core/ut_suite_tag_filter.pks index 787c72603..37ba19111 100644 --- a/source/core/ut_suite_tag_filter.pks +++ b/source/core/ut_suite_tag_filter.pks @@ -31,7 +31,7 @@ create or replace package ut_suite_tag_filter authid definer is * and return a list of elements in Reverse Polish Notation ( postfix ) * As part of execution it will validate expression. */ - function shunt_logical_expression(a_tags in varchar2) return ut_varchar2_list; + function shunt_logical_expression(a_tags in ut_varchar2_list) return ut_varchar2_list; /* * Function that converts postfix notation into infix and creating a string of sql filter diff --git a/test/ut3_tester/core/test_ut_suite_tag_filter.pkb b/test/ut3_tester/core/test_ut_suite_tag_filter.pkb index a1d0be1f7..a396c6d25 100644 --- a/test/ut3_tester/core/test_ut_suite_tag_filter.pkb +++ b/test/ut3_tester/core/test_ut_suite_tag_filter.pkb @@ -3,24 +3,65 @@ create or replace package body test_ut_suite_tag_filter is procedure test_conversion_to_rpn is l_postfix ut3_develop.ut_varchar2_list; l_postfix_string varchar2(4000); + l_input_token ut3_develop.ut_varchar2_list; begin - l_postfix := ut3_develop.ut_suite_tag_filter.shunt_logical_expression('A'); + l_input_token := ut3_develop.ut_suite_tag_filter.tokenize_tags_string('A'); + l_postfix := ut3_develop.ut_suite_tag_filter.shunt_logical_expression(l_input_token); l_postfix_string := ut3_develop.ut_utils.table_to_clob(l_postfix,''); ut.expect(l_postfix_string).to_equal('A'); - l_postfix := ut3_develop.ut_suite_tag_filter.shunt_logical_expression('A|B'); + l_input_token := ut3_develop.ut_suite_tag_filter.tokenize_tags_string('A|B'); + l_postfix := ut3_develop.ut_suite_tag_filter.shunt_logical_expression(l_input_token); l_postfix_string := ut3_develop.ut_utils.table_to_clob(l_postfix,''); ut.expect(l_postfix_string).to_equal('AB|'); - l_postfix := ut3_develop.ut_suite_tag_filter.shunt_logical_expression('(a|b)|c&d'); + l_input_token := ut3_develop.ut_suite_tag_filter.tokenize_tags_string('(a|b)|c&d'); + l_postfix := ut3_develop.ut_suite_tag_filter.shunt_logical_expression(l_input_token); l_postfix_string := ut3_develop.ut_utils.table_to_clob(l_postfix,''); ut.expect(l_postfix_string).to_equal('ab|cd&|'); - l_postfix := ut3_develop.ut_suite_tag_filter.shunt_logical_expression('!a|b'); + l_input_token := ut3_develop.ut_suite_tag_filter.tokenize_tags_string('!a|b'); + l_postfix := ut3_develop.ut_suite_tag_filter.shunt_logical_expression(l_input_token); l_postfix_string := ut3_develop.ut_utils.table_to_clob(l_postfix,''); ut.expect(l_postfix_string).to_equal('a!b|'); end; + procedure test_conversion_opr_by_opr is + l_postfix ut3_develop.ut_varchar2_list; + l_input_token ut3_develop.ut_varchar2_list; + begin + l_input_token := ut3_develop.ut_varchar2_list('A','B'); + l_postfix := ut3_develop.ut_suite_tag_filter.shunt_logical_expression(l_input_token); + ut.fail('Expected exception but nothing was raised'); + end; + + procedure test_conversion_oprd_by_opd is + l_postfix ut3_develop.ut_varchar2_list; + l_input_token ut3_develop.ut_varchar2_list; + begin + l_input_token := ut3_develop.ut_varchar2_list('|','|'); + l_postfix := ut3_develop.ut_suite_tag_filter.shunt_logical_expression(l_input_token); + ut.fail('Expected exception but nothing was raised'); + end; + + procedure test_conversion_lb_by_oper is + l_postfix ut3_develop.ut_varchar2_list; + l_input_token ut3_develop.ut_varchar2_list; + begin + l_input_token := ut3_develop.ut_varchar2_list('(','A','|','B',')','('); + l_postfix := ut3_develop.ut_suite_tag_filter.shunt_logical_expression(l_input_token); + ut.fail('Expected exception but nothing was raised'); + end; + + procedure test_conversion_rb_by_oprd is + l_postfix ut3_develop.ut_varchar2_list; + l_input_token ut3_develop.ut_varchar2_list; + begin + l_input_token := ut3_develop.ut_varchar2_list(')','A'); + l_postfix := ut3_develop.ut_suite_tag_filter.shunt_logical_expression(l_input_token); + ut.fail('Expected exception but nothing was raised'); + end; + procedure conv_from_rpn_to_sql_filter is l_postfix_rpn ut3_develop.ut_varchar2_list; l_infix_string varchar2(4000); diff --git a/test/ut3_tester/core/test_ut_suite_tag_filter.pks b/test/ut3_tester/core/test_ut_suite_tag_filter.pks index 093fab856..6d0b9ff69 100644 --- a/test/ut3_tester/core/test_ut_suite_tag_filter.pks +++ b/test/ut3_tester/core/test_ut_suite_tag_filter.pks @@ -3,9 +3,29 @@ create or replace package test_ut_suite_tag_filter is --%suite(ut_suite_tag_filter) --%suitepath(utplsql.ut3_tester.core) + --%context( Conversion to Reverse Polish Notation) + --%test( Test conversion of expression into Reverse Polish Notation) procedure test_conversion_to_rpn; + --%test( Operator is followed by operator) + --%throws(ut3_develop.ut_utils.gc_invalid_tag_expression) + procedure test_conversion_opr_by_opr; + + --%test( Operand is followed by operand) + --%throws(ut3_develop.ut_utils.gc_invalid_tag_expression) + procedure test_conversion_oprd_by_opd; + + --%test( Left Bracket is followed by operator) + --%throws(ut3_develop.ut_utils.gc_invalid_tag_expression) + procedure test_conversion_lb_by_oper; + + --%test( Right Bracket is followed by operand) + --%throws(ut3_develop.ut_utils.gc_invalid_tag_expression) + procedure test_conversion_rb_by_oprd; + + --%endcontext + --%test( Test conversion of expression from Reverse Polish Notation into custom where filter for SQL) procedure conv_from_rpn_to_sql_filter; From 01e53646ba0e37b02ca476832061053e5aa5e786 Mon Sep 17 00:00:00 2001 From: Lukasz Wasylow Date: Fri, 14 Apr 2023 23:21:35 -0700 Subject: [PATCH 50/77] Update tests and code --- source/core/ut_suite_tag_filter.pkb | 36 +++++++++---------- source/core/ut_suite_tag_filter.pks | 9 ++++- .../core/test_ut_suite_tag_filter.pkb | 23 ++++++------ .../core/test_ut_suite_tag_filter.pks | 4 +-- 4 files changed, 38 insertions(+), 34 deletions(-) diff --git a/source/core/ut_suite_tag_filter.pkb b/source/core/ut_suite_tag_filter.pkb index d5df54212..df125091d 100644 --- a/source/core/ut_suite_tag_filter.pkb +++ b/source/core/ut_suite_tag_filter.pkb @@ -22,7 +22,9 @@ create or replace package body ut_suite_tag_filter is gc_operators constant ut_varchar2_list := ut_varchar2_list('|','&','!'); gc_unary_operators constant ut_varchar2_list := ut_varchar2_list('!'); -- right side associative operator gc_binary_operators constant ut_varchar2_list := ut_varchar2_list('|','&'); -- left side associative operator - + gc_tags_column_name constant varchar2(250) := 'tags'; + gc_exception_msg constant varchar2(200) := 'Invalid tag expression'; + type t_precedence_table is table of number index by varchar2(1); g_precedence t_precedence_table; @@ -101,7 +103,7 @@ create or replace package body ut_suite_tag_filter is l_token := a_tags(l_idx); if (l_token member of gc_operators and l_token member of gc_binary_operators) then if not(l_expect_operator) then - raise_application_error(ut_utils.gc_invalid_tag_expression, 'Invalid Tag expression'); + raise_application_error(ut_utils.gc_invalid_tag_expression, gc_exception_msg); end if; while l_operator_stack.top > 0 and (g_precedence(l_operator_stack.peek) > g_precedence(l_token)) loop l_rnp_tokens.extend; @@ -112,21 +114,21 @@ create or replace package body ut_suite_tag_filter is l_expect_operator:= false; elsif (l_token member of gc_operators and l_token member of gc_unary_operators) then if not(l_expect_operand) then - raise_application_error(ut_utils.gc_invalid_tag_expression, 'Invalid Tag expression'); + raise_application_error(ut_utils.gc_invalid_tag_expression, gc_exception_msg); end if; l_operator_stack.push(a_tags(l_idx)); l_expect_operand := true; l_expect_operator:= false; elsif l_token = '(' then if not(l_expect_operand) then - raise_application_error(ut_utils.gc_invalid_tag_expression, 'Invalid Tag expression'); + raise_application_error(ut_utils.gc_invalid_tag_expression, gc_exception_msg); end if; l_operator_stack.push(a_tags(l_idx)); l_expect_operand := true; l_expect_operator:= false; elsif l_token = ')' then if not(l_expect_operator) then - raise_application_error(ut_utils.gc_invalid_tag_expression, 'Invalid Tag expression'); + raise_application_error(ut_utils.gc_invalid_tag_expression, gc_exception_msg); end if; while l_operator_stack.peek <> '(' loop l_rnp_tokens.extend; @@ -137,7 +139,7 @@ create or replace package body ut_suite_tag_filter is l_expect_operator:= true; else if not(l_expect_operand) then - raise_application_error(ut_utils.gc_invalid_tag_expression, 'Invalid Tag expression'); + raise_application_error(ut_utils.gc_invalid_tag_expression, gc_exception_msg); end if; l_rnp_tokens.extend; l_rnp_tokens(l_rnp_tokens.last) :=l_token; @@ -150,7 +152,7 @@ create or replace package body ut_suite_tag_filter is while l_operator_stack.peek is not null loop if l_operator_stack.peek in ('(',')') then - raise_application_error(ut_utils.gc_invalid_tag_expression, 'Invalid Tag expression'); + raise_application_error(ut_utils.gc_invalid_tag_expression, gc_exception_msg); end if; l_rnp_tokens.extend; l_rnp_tokens(l_rnp_tokens.last):=l_operator_stack.pop; @@ -159,13 +161,13 @@ create or replace package body ut_suite_tag_filter is return l_rnp_tokens; end shunt_logical_expression; - function conv_postfix_to_infix_sql(a_postfix_exp in ut_varchar2_list) + function conv_postfix_to_infix_sql(a_postfix_exp in ut_varchar2_list,a_tags_column_name in varchar2) return varchar2 is l_infix_stack ut_stack := ut_stack(); l_right_side varchar2(32767); l_left_side varchar2(32767); l_infix_exp varchar2(32767); - l_member_token varchar2(20) := ' member of tags'; + l_member_token varchar2(20) := ' member of '||a_tags_column_name; l_idx pls_integer; begin l_idx := a_postfix_exp.first; @@ -173,9 +175,6 @@ create or replace package body ut_suite_tag_filter is --If token is operand but also single tag if regexp_count(a_postfix_exp(l_idx),'[!()|&]') = 0 then l_infix_stack.push(q'[']'||a_postfix_exp(l_idx)||q'[']'||l_member_token); - --If token is operand but containing other expressions - elsif a_postfix_exp(l_idx) not member of gc_operators then - l_infix_stack.push(a_postfix_exp(l_idx)); --If token is unary operator not elsif a_postfix_exp(l_idx) member of gc_unary_operators then l_right_side := l_infix_stack.pop; @@ -197,10 +196,9 @@ create or replace package body ut_suite_tag_filter is function create_where_filter(a_tags varchar2 ) return varchar2 is l_tags varchar2(4000); - l_tokenized_tags ut_varchar2_list; begin l_tags := replace(replace_legacy_tag_notation(a_tags),' '); - l_tags := conv_postfix_to_infix_sql(shunt_logical_expression(tokenize_tags_string(l_tags))); + l_tags := conv_postfix_to_infix_sql(shunt_logical_expression(tokenize_tags_string(l_tags)),gc_tags_column_name); l_tags := replace(l_tags, '|',' or '); l_tags := replace(l_tags ,'&',' and '); l_tags := replace(l_tags ,'!','not'); @@ -224,7 +222,7 @@ create or replace package body ut_suite_tag_filter is q'[ with suites_mv as ( - select c.id,value(c) as obj,c.path as path,c.self_type,c.object_owner,c.tags + select c.id,value(c) as obj,c.path as path,c.self_type,c.object_owner,c.tags as ]'||gc_tags_column_name||q'[ from table(:suite_items) c ), suites_matching_expr as ( @@ -234,17 +232,17 @@ with and ]'||l_tags||q'[ ), tests_matching_expr as ( - select c.id,c.path as path,c.self_type,c.object_owner,c.tags - from suites_mv c where c.self_type in ('UT_TEST') + select c.id,c.path as path,c.self_type,c.object_owner,c.tags as ]'||gc_tags_column_name||q'[ + from suites_mv c where c.self_type in ('UT_TEST') and ]'||l_tags||q'[ ), tests_with_tags_inh_from_suite as ( - select c.id,c.self_type,c.path,c.tags multiset union distinct t.tags tags,c.object_owner + select c.id,c.self_type,c.path,c.tags multiset union distinct t.tags as ]'||gc_tags_column_name||q'[ ,c.object_owner from suites_mv c join suites_matching_expr t on (c.path||'.' like t.path || '.%' /*all descendants and self*/ and c.object_owner = t.object_owner) ), tests_with_tags_prom_to_suite as ( - select c.id,c.self_type,c.path,c.tags multiset union distinct t.tags tags,c.object_owner + select c.id,c.self_type,c.path,c.tags multiset union distinct t.tags as ]'||gc_tags_column_name||q'[ ,c.object_owner from suites_mv c join tests_matching_expr t on (t.path||'.' like c.path || '.%' /*all ancestors and self*/ and c.object_owner = t.object_owner) ) diff --git a/source/core/ut_suite_tag_filter.pks b/source/core/ut_suite_tag_filter.pks index 37ba19111..e824ae275 100644 --- a/source/core/ut_suite_tag_filter.pks +++ b/source/core/ut_suite_tag_filter.pks @@ -37,8 +37,15 @@ create or replace package ut_suite_tag_filter authid definer is * Function that converts postfix notation into infix and creating a string of sql filter * that checking a tags collections for tags according to posted logic. */ - function conv_postfix_to_infix_sql(a_postfix_exp in ut_varchar2_list) return varchar2; + function conv_postfix_to_infix_sql(a_postfix_exp in ut_varchar2_list,a_tags_column_name in varchar2) + return varchar2; + /* + * Generates a part where clause sql + */ + function create_where_filter(a_tags varchar2) + return varchar2; + function apply( a_unfiltered_rows ut_suite_cache_rows, a_tags varchar2 := null diff --git a/test/ut3_tester/core/test_ut_suite_tag_filter.pkb b/test/ut3_tester/core/test_ut_suite_tag_filter.pkb index a396c6d25..edfb27cfc 100644 --- a/test/ut3_tester/core/test_ut_suite_tag_filter.pkb +++ b/test/ut3_tester/core/test_ut_suite_tag_filter.pkb @@ -62,21 +62,20 @@ create or replace package body test_ut_suite_tag_filter is ut.fail('Expected exception but nothing was raised'); end; - procedure conv_from_rpn_to_sql_filter is - l_postfix_rpn ut3_develop.ut_varchar2_list; - l_infix_string varchar2(4000); + procedure conv_from_tag_to_sql_filter is + l_sql_filter varchar2(4000); begin - l_postfix_rpn := ut3_develop.ut_varchar2_list('A'); - l_infix_string := ut3_develop.ut_suite_tag_filter.conv_postfix_to_infix_sql(l_postfix_rpn); - ut.expect(l_infix_string).to_equal(q'['A' member of tags]'); + l_sql_filter := ut3_develop.ut_suite_tag_filter.create_where_filter('test1'); + ut.expect(l_sql_filter).to_equal(q'['test1' member of tags]'); - l_postfix_rpn := ut3_develop.ut_varchar2_list('A','B','|'); - l_infix_string := ut3_develop.ut_suite_tag_filter.conv_postfix_to_infix_sql(l_postfix_rpn); - ut.expect(l_infix_string).to_equal(q'[('A' member of tags|'B' member of tags)]'); + l_sql_filter := ut3_develop.ut_suite_tag_filter.create_where_filter('test1|test2'); + ut.expect(l_sql_filter).to_equal(q'[('test1' member of tags or 'test2' member of tags)]'); - l_postfix_rpn := ut3_develop.ut_varchar2_list('a','b','!','|'); - l_infix_string := ut3_develop.ut_suite_tag_filter.conv_postfix_to_infix_sql(l_postfix_rpn); - ut.expect(l_infix_string).to_equal(q'[('a' member of tags|!('b' member of tags))]'); + l_sql_filter := ut3_develop.ut_suite_tag_filter.create_where_filter('test1|!test2'); + ut.expect(l_sql_filter).to_equal(q'[('test1' member of tags or not('test2' member of tags))]'); + + l_sql_filter := ut3_develop.ut_suite_tag_filter.create_where_filter('test1&!test2'); + ut.expect(l_sql_filter).to_equal(q'[('test1' member of tags and not('test2' member of tags))]'); end; end test_ut_suite_tag_filter; diff --git a/test/ut3_tester/core/test_ut_suite_tag_filter.pks b/test/ut3_tester/core/test_ut_suite_tag_filter.pks index 6d0b9ff69..0f84b751b 100644 --- a/test/ut3_tester/core/test_ut_suite_tag_filter.pks +++ b/test/ut3_tester/core/test_ut_suite_tag_filter.pks @@ -26,8 +26,8 @@ create or replace package test_ut_suite_tag_filter is --%endcontext - --%test( Test conversion of expression from Reverse Polish Notation into custom where filter for SQL) - procedure conv_from_rpn_to_sql_filter; + --%test( Test conversion of expression from tag into custom where filter for SQL) + procedure conv_from_tag_to_sql_filter; end test_ut_suite_tag_filter; / From dc0b4a60dfa799b360245bc7c39480e11861f564 Mon Sep 17 00:00:00 2001 From: Lukasz Wasylow Date: Mon, 17 Apr 2023 19:37:03 -0700 Subject: [PATCH 51/77] Addressing changes via PR review. --- docs/userguide/annotations.md | 40 +----------- docs/userguide/running-unit-tests.md | 90 +++++++++++++++++++------- source/core/ut_suite_cache_manager.pkb | 14 +--- source/core/ut_suite_cache_manager.pks | 9 ++- source/core/ut_suite_manager.pkb | 5 +- 5 files changed, 75 insertions(+), 83 deletions(-) diff --git a/docs/userguide/annotations.md b/docs/userguide/annotations.md index 552b02b52..17fb7d1f2 100644 --- a/docs/userguide/annotations.md +++ b/docs/userguide/annotations.md @@ -1617,46 +1617,8 @@ or Tags are defined as a comma separated list within the `--%tags` annotation. When a suite/context is tagged, all of its children will automatically inherit the tag and get executed along with the parent, unless they are excluded explicitly at runtime with a negated tag expression. -Parent suite tests are not executed, but a suitepath hierarchy is kept. +See [running unit tests](running-unit-tests.md) for more information on using tags to filter test suites that are to be executed. -Sample test suite package with tags. -```sql linenums="1" -create or replace package ut_sample_test is - - --%suite(Sample Test Suite) - --%tags(api) - - --%test(Compare Ref Cursors) - --%tags(complex,fast) - procedure ut_refcursors1; - - --%test(Run equality test) - --%tags(simple,fast) - procedure ut_test; - -end ut_sample_test; -/ - -create or replace package body ut_sample_test is - - procedure ut_refcursors1 is - v_actual sys_refcursor; - v_expected sys_refcursor; - begin - open v_expected for select 1 as test from dual; - open v_actual for select 2 as test from dual; - - ut.expect(v_actual).to_equal(v_expected); - end; - - procedure ut_test is - begin - ut.expect(1).to_equal(0); - end; - -end ut_sample_test; -/ -``` #### Tag naming convention Tags must follow the below naming convention: diff --git a/docs/userguide/running-unit-tests.md b/docs/userguide/running-unit-tests.md index 9abc7c964..7597ae72a 100644 --- a/docs/userguide/running-unit-tests.md +++ b/docs/userguide/running-unit-tests.md @@ -323,7 +323,7 @@ Multiple tags are separated by comma. ### Tag Expressions -Tag expressions are boolean expressions with the operators !, & and |. In addition, ( and ) can be used to adjust for operator precedence. +Tag expressions are boolean expressions created by combining tags with the `!`, `&`, `|` operators. Tag expressions can be grouped using `(` and `)` braces. Grouping tag expressions affects operator precedence. | Operator | Meaning | | -------- | --------| @@ -338,18 +338,76 @@ If you are tagging your tests across multiple dimensions, tag expressions help y | -------- | --------| | product | all tests for product | | catalog \| shipping | all tests for catalog plus all tests for shipping | -| catalog & shipping | all tests for the intersection between catalog and shipping | -| product & !end-to-end | all tests for product, but not the end-to-end tests | +| catalog & shipping | all tests that are tagged with both `catalog` and `shipping` tags | +| product & !end-to-end | all tests tagged `product`, except the tests tagged `end-to-end` | | (micro \| integration) & (product \| shipping) | all micro or integration tests for product or shipping | -Execution of the test is done by using the parameter `a_tags` with tag expressions +Taking the last expression above `(micro | integration) & (product | shipping)` +| --%tags |included in run | +| -------- | --------| +| micro | no | +| integration | no | +| micro | no | +| product | no | +| shipping | no | +| micro | no | +| micro, integration | no | +| product, shipping | no | +| micro, product | yes | +| micro, shipping | yes | +| integration, product | yes | +| integration, shipping | yes | +| integration, micro, shipping | yes | +| integration, micro, product | yes | +| integration, shipping ,product | yes | +| micro, shipping ,product | yes | +| integration, micro, shipping ,product | yes | + + +### Sample execution of test with tags. + +Execution of the test with tag expressions is done using the parameter `a_tags`. +Given a test package `ut_sample_test` defined below ```sql linenums="1" -select * from table(ut.run(a_tags => 'fast|!complex')); +create or replace package ut_sample_test is + + --%suite(Sample Test Suite) + --%tags(api) + + --%test(Compare Ref Cursors) + --%tags(complex,fast) + procedure ut_refcursors1; + + --%test(Run equality test) + --%tags(simple,fast) + procedure ut_test; + +end ut_sample_test; +/ + +create or replace package body ut_sample_test is + + procedure ut_refcursors1 is + v_actual sys_refcursor; + v_expected sys_refcursor; + begin + open v_expected for select 1 as test from dual; + open v_actual for select 2 as test from dual; + + ut.expect(v_actual).to_equal(v_expected); + end; + + procedure ut_test is + begin + ut.expect(1).to_equal(0); + end; + +end ut_sample_test; +/ ``` -The above call will execute all tests from `ut_sample_test` package as the whole suite is tagged with `api` because a suite meet expression condition. ```sql linenums="1" select * from table(ut.run(a_path => 'ut_sample_test',a_tags => 'api')); @@ -357,9 +415,9 @@ select * from table(ut.run(a_path => 'ut_sample_test',a_tags => 'api')); The above call will execute all tests from `ut_sample_test` package as the whole suite is tagged with `api` ```sql linenums="1" -select * from table(ut.run(a_tags => 'complex')); +select * from table(ut.run(a_tags => 'fast&complex')); ``` -The above call will execute only the `ut_sample_test.ut_refcursors1` test, as only the test `ut_refcursors1` is tagged with `complex` +The above call will execute only the `ut_sample_test.ut_refcursors1` test, as only the test `ut_refcursors1` is tagged with `complex` and `fast` ```sql linenums="1" select * from table(ut.run(a_tags => 'fast')); @@ -376,25 +434,13 @@ Examples (based on above sample test suite) select * from table(ut.run(a_tags => '(api|fast)&!complex')); ``` -which is equivalent of legacy calling: - -```sql linenums="1" -select * from table(ut.run(a_tags => 'api,fast,-complex')); -``` - or ```sql linenums="1" -select * from table(ut.run(a_tags => '(api|fast)&(!complex&!test1)')); -``` - -which is equivalent of legacy calling: - -```sql linenums="1" -select * from table(ut.run(a_tags => 'api,fast,-complex,-test1')); +select * from table(ut.run(a_tags => '(api|fast)&!complex&!test1')); ``` -The above call will execute all suites/contexts/tests that are marked with any of tags `api` or `fast` except those suites/contexts/tests that are marked as `complex`. +The above call will execute all suites/contexts/tests that are marked with any of tags `api` or `fast` except those suites/contexts/tests that are marked as `complex` and except those suites/contexts/tests that are marked as `test1`. Given the above example package `ut_sample_test`, only `ut_sample_test.ut_test` will be executed. diff --git a/source/core/ut_suite_cache_manager.pkb b/source/core/ut_suite_cache_manager.pkb index 3bb679517..680624f02 100644 --- a/source/core/ut_suite_cache_manager.pkb +++ b/source/core/ut_suite_cache_manager.pkb @@ -285,18 +285,6 @@ create or replace package body ut_suite_cache_manager is return l_results; end; - function get_cached_suites( - a_schema_paths ut_path_items, - a_random_seed positive := null - ) return ut_suite_cache_rows is - l_suite_items ut_suite_cache_rows := ut_suite_cache_rows(); - l_schema_paths ut_path_items; - begin - l_schema_paths := a_schema_paths; - l_suite_items := get_suite_items(a_schema_paths); - return l_suite_items; - end; - function get_schema_parse_time(a_schema_name varchar2) return timestamp result_cache is l_cache_parse_time timestamp; begin @@ -443,7 +431,7 @@ create or replace package body ut_suite_cache_manager is a_schema_paths ut_path_items ) return ut_suite_cache_rows is begin - return get_cached_suite_rows(get_cached_suites( a_schema_paths )); + return get_cached_suite_rows(get_suite_items(a_schema_paths)); end; function get_suite_items_info( diff --git a/source/core/ut_suite_cache_manager.pks b/source/core/ut_suite_cache_manager.pks index c217abeed..81b1e0136 100644 --- a/source/core/ut_suite_cache_manager.pks +++ b/source/core/ut_suite_cache_manager.pks @@ -58,11 +58,6 @@ create or replace package ut_suite_cache_manager authid definer is a_suites_filtered ut_suite_cache_rows ) return ut_suite_cache_rows; - function get_cached_suites( - a_schema_paths ut_path_items, - a_random_seed positive := null - ) return ut_suite_cache_rows; - function get_schema_paths(a_paths in ut_varchar2_list) return ut_path_items; /* @@ -77,6 +72,10 @@ create or replace package ut_suite_cache_manager authid definer is function get_suite_items_info( a_suite_cache_items ut_suite_cache_rows ) return ut_suite_items_info; + + function get_suite_items ( + a_schema_paths ut_path_items + ) return ut_suite_cache_rows; /* * Retrieves list of cached suite packages. diff --git a/source/core/ut_suite_manager.pkb b/source/core/ut_suite_manager.pkb index f8a2e002c..c46ba66b8 100644 --- a/source/core/ut_suite_manager.pkb +++ b/source/core/ut_suite_manager.pkb @@ -360,10 +360,7 @@ create or replace package body ut_suite_manager is l_filtered_rows ut_suite_cache_rows; l_result t_cached_suites_cursor; begin - l_unfiltered_rows := ut_suite_cache_manager.get_cached_suites( - a_schema_paths, - a_random_seed - ); + l_unfiltered_rows := ut_suite_cache_manager.get_suite_items(a_schema_paths); l_tag_filter_applied := ut_suite_tag_filter.apply(l_unfiltered_rows,a_tags); l_filtered_rows := get_filtered_cursor(ut_suite_cache_manager.get_cached_suite_rows(l_tag_filter_applied),a_skip_all_objects); From ef1c02b3eee5b3a5a71076f1db24a58765cddbda Mon Sep 17 00:00:00 2001 From: Lukasz Wasylow Date: Tue, 18 Apr 2023 15:36:11 -0700 Subject: [PATCH 52/77] Update docs --- docs/userguide/running-unit-tests.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/docs/userguide/running-unit-tests.md b/docs/userguide/running-unit-tests.md index 7597ae72a..fb9a9f602 100644 --- a/docs/userguide/running-unit-tests.md +++ b/docs/userguide/running-unit-tests.md @@ -440,7 +440,13 @@ or select * from table(ut.run(a_tags => '(api|fast)&!complex&!test1')); ``` -The above call will execute all suites/contexts/tests that are marked with any of tags `api` or `fast` except those suites/contexts/tests that are marked as `complex` and except those suites/contexts/tests that are marked as `test1`. +which is equivalent of exclusion on whole expression + +```sql linenums="1" +select * from table(ut.run(a_tags => '(api|fast)&!(complex|test1)')); +``` + +The above calls will execute all suites/contexts/tests that are marked with any of tags `api` or `fast` except those suites/contexts/tests that are marked as `complex` and except those suites/contexts/tests that are marked as `test1`. Given the above example package `ut_sample_test`, only `ut_sample_test.ut_test` will be executed. From 1551ea5583152a0d0b726005906aa6e51f42c38a Mon Sep 17 00:00:00 2001 From: Lukasz Wasylow Date: Tue, 25 Apr 2023 09:30:40 -0700 Subject: [PATCH 53/77] Adding any and none --- docs/userguide/annotations.md | 1 + docs/userguide/running-unit-tests.md | 75 ++++++++++++ source/core/ut_suite_tag_filter.pkb | 15 ++- test/ut3_tester_helper/run_helper.pkb | 160 +++++++++++++++++++++++++- test/ut3_user/api/test_ut_run.pkb | 25 ++++ 5 files changed, 271 insertions(+), 5 deletions(-) diff --git a/docs/userguide/annotations.md b/docs/userguide/annotations.md index 17fb7d1f2..4ec6de59d 100644 --- a/docs/userguide/annotations.md +++ b/docs/userguide/annotations.md @@ -1634,6 +1634,7 @@ Tags must follow the below naming convention: - tag cannot start with a dash, e.g. `-some-stuff` is **not** a valid tag - tag cannot contain spaces, e.g. `test of batch`. To create a multi-word tag use underscores or dashes, e.g. `test_of_batch`, `test-of-batch` - leading and trailing spaces are ignored in tag name, e.g. `--%tags( tag1 , tag2 )` becomes `tag1` and `tag2` tag names +- tag cannot be one of two reserved words : `none` and `any`, any tags with that will not be considered. ### Suitepath diff --git a/docs/userguide/running-unit-tests.md b/docs/userguide/running-unit-tests.md index fb9a9f602..b3a974d53 100644 --- a/docs/userguide/running-unit-tests.md +++ b/docs/userguide/running-unit-tests.md @@ -325,6 +325,8 @@ Multiple tags are separated by comma. Tag expressions are boolean expressions created by combining tags with the `!`, `&`, `|` operators. Tag expressions can be grouped using `(` and `)` braces. Grouping tag expressions affects operator precedence. +Two special expressions are supported, `any` and `none`, which select all tests with any tags at all, and all tests without any tags, respectively. These special expressions may be combined with other expressions just like normal tags. When using `none` be aware that if the suite is tagged it will exclude any tests and children belonging to that suite. + | Operator | Meaning | | -------- | --------| | ! | not | @@ -450,6 +452,79 @@ The above calls will execute all suites/contexts/tests that are marked with any Given the above example package `ut_sample_test`, only `ut_sample_test.ut_test` will be executed. +### Sample execution with `any` and `none` + +Given a sample test package: + +```sql linenums="1" +create or replace package ut_sample_test is + + --%suite(Sample Test Suite) + + --%test(Compare Ref Cursors) + --%tags(complex,fast) + procedure ut_refcursors1; + + --%test(Run equality test) + --%tags(simple,fast) + procedure ut_test; + + --%test(Run equality test no tag) + procedure ut_test_no_tag; + +end ut_sample_test; +/ + +create or replace package body ut_sample_test is + + procedure ut_refcursors1 is + v_actual sys_refcursor; + v_expected sys_refcursor; + begin + open v_expected for select 1 as test from dual; + open v_actual for select 2 as test from dual; + + ut.expect(v_actual).to_equal(v_expected); + end; + + procedure ut_test is + begin + ut.expect(1).to_equal(0); + end; + + procedure ut_test_no_tag is + begin + ut.expect(1).to_equal(0); + end; + +end ut_sample_test; +/ +``` + +```sql linenums="1" +select * from table(ut.run(a_path => 'ut_sample_test',a_tags => 'none')); +``` + +The above call will execute tests `ut_test_no_tag` + +```sql linenums="1" +select * from table(ut.run(a_path => 'ut_sample_test',a_tags => 'any')); +``` + +The above call will execute tests `ut_test` and `ut_refcursors1` + +```sql linenums="1" +select * from table(ut.run(a_path => 'ut_sample_test',a_tags => 'none|simple')); +``` + +The above call will execute tests `ut_test_no_tag` and `ut_test` + +```sql linenums="1" +select * from table(ut.run(a_tags => 'none|!simple')); +``` + +The above call will execute tests `ut_test_no_tag` and `ut_refcursors1` + ## Keeping uncommitted data after test-run utPLSQL by default runs tests in autonomous transaction and performs automatic rollback to assure that tests do not impact one-another and do not have impact on the current session in your IDE. diff --git a/source/core/ut_suite_tag_filter.pkb b/source/core/ut_suite_tag_filter.pkb index df125091d..2d9436301 100644 --- a/source/core/ut_suite_tag_filter.pkb +++ b/source/core/ut_suite_tag_filter.pkb @@ -22,9 +22,10 @@ create or replace package body ut_suite_tag_filter is gc_operators constant ut_varchar2_list := ut_varchar2_list('|','&','!'); gc_unary_operators constant ut_varchar2_list := ut_varchar2_list('!'); -- right side associative operator gc_binary_operators constant ut_varchar2_list := ut_varchar2_list('|','&'); -- left side associative operator + gc_reserved_tag_words constant ut_varchar2_list := ut_varchar2_list('none','any'); gc_tags_column_name constant varchar2(250) := 'tags'; gc_exception_msg constant varchar2(200) := 'Invalid tag expression'; - + type t_precedence_table is table of number index by varchar2(1); g_precedence t_precedence_table; @@ -172,8 +173,17 @@ create or replace package body ut_suite_tag_filter is begin l_idx := a_postfix_exp.first; while ( l_idx is not null) loop + --If the token we got is a none or any keyword + if a_postfix_exp(l_idx) member of gc_reserved_tag_words then + + l_infix_stack.push( + case + when a_postfix_exp(l_idx) = 'none' then '('||a_tags_column_name||' is empty or '||a_tags_column_name||' is null)' + else a_tags_column_name||' is not empty' + end + ); --If token is operand but also single tag - if regexp_count(a_postfix_exp(l_idx),'[!()|&]') = 0 then + elsif regexp_count(a_postfix_exp(l_idx),'[!()|&]') = 0 then l_infix_stack.push(q'[']'||a_postfix_exp(l_idx)||q'[']'||l_member_token); --If token is unary operator not elsif a_postfix_exp(l_idx) member of gc_unary_operators then @@ -256,7 +266,6 @@ with where ]'||l_tags||q'[ ) ) t where c.id = t.id and r_num = 1 ]'; - execute immediate l_sql bulk collect into l_suite_tags using a_suite_items; return l_suite_tags; end; diff --git a/test/ut3_tester_helper/run_helper.pkb b/test/ut3_tester_helper/run_helper.pkb index c73564798..6289b1642 100644 --- a/test/ut3_tester_helper/run_helper.pkb +++ b/test/ut3_tester_helper/run_helper.pkb @@ -431,12 +431,162 @@ create or replace package body run_helper is end test_tag_pkg_3; ]'; + execute immediate q'[create or replace package suite1_level1_pkg is + + --%suite(suite1_level1) + --%suitepath(any_none) + --%rollback(manual) + + --%test(Test 1 from Suite1 on level 1) + --%tags(suite1,level1,test1,test1_level1) + procedure test1_level1; + + --%test(Test 2 from Suite1 on level 1) + procedure test2_level1; + + end suite1_level1_pkg; + ]'; + + execute immediate q'[create or replace package body suite1_level1_pkg is + procedure test1_level1 is + begin + dbms_output.put_line('suite1_level1_pkg.test1_level1 executed'); + end; + procedure test2_level1 is + begin + dbms_output.put_line('suite1_level1_pkg.test2_level1 executed'); + end; + end suite1_level1_pkg; + ]'; + + execute immediate q'[create or replace package suite1_1_level2_pkg is + + --%suite(suite1_1_level2) + --%suitepath(any_none.suite1_level1) + --%rollback(manual) + + --%test(Test 1 from Suite1_2 on level 2) + --%tags(level2,test1,test1_level2) + procedure suite1_1_test1_level2; + + --%test(Test 2 from Suite1_2 on level 2) + procedure suite1_1_test2_level2; + + end suite1_1_level2_pkg; + ]'; + + execute immediate q'[create or replace package body suite1_1_level2_pkg is + procedure suite1_1_test1_level2 is + begin + dbms_output.put_line('suite1_1_level2_pkg.suite1_1_test1_level2 executed'); + end; + procedure suite1_1_test2_level2 is + begin + dbms_output.put_line('suite1_1_level2_pkg.suite1_1_test2_level2 executed'); + end; + end suite1_1_level2_pkg; + ]'; + + execute immediate q'[create or replace package suite1_2_level2_pkg is + + --%suite(suite1_2_level2) + --%tags(level2,suite1_2,suites) + --%suitepath(any_none.suite1_level1) + --%rollback(manual) + + --%test(Test 1 from Suite1_2 on level 2) + procedure suite1_2_test1_level2; + + --%test(Test 2 from Suite1_2 on level 2) + --%tags(level2,test2,test2_level2) + procedure suite1_2_test2_level1; + + end suite1_2_level2_pkg; + ]'; + + execute immediate q'[create or replace package body suite1_2_level2_pkg is + procedure suite1_2_test1_level2 is + begin + dbms_output.put_line('suite1_2_level2_pkg.suite1_2_test1_level2 executed'); + end; + procedure suite1_2_test2_level1 is + begin + dbms_output.put_line('suite1_2_level2_pkg.suite1_2_test2_level1 executed'); + end; + end suite1_2_level2_pkg; + ]'; + + execute immediate q'[create or replace package suite2_level1_pkg is + + --%suite(suite2_level1) + --%tags(level1,suite2,suites) + --%suitepath(any_none) + --%rollback(manual) + + --%test(Test 1 from Suite1 on level 1) + --%tags(suite2,level1,test1,test1_level1) + procedure test1_level1; + + --%test(Test 2 from Suite1 on level 1) + procedure test2_level1; + + end suite2_level1_pkg; + ]'; + + execute immediate q'[create or replace package body suite2_level1_pkg is + procedure test1_level1 is + begin + dbms_output.put_line('suite2_level1_pkg.test1_level1 executed'); + end; + procedure test2_level1 is + begin + dbms_output.put_line('suite2_level1_pkg.test2_level1 executed'); + end; + end suite2_level1_pkg; + ]'; + + execute immediate q'[create or replace package suite2_2_level2_pkg is + + --%suite(suite2_2_level2) + --%tags(level2,suite2_2,suites) + --%suitepath(any_none.suite2_level1) + --%rollback(manual) + + --%test(Test 1 from Suite2_2 on level 2) + procedure suite2_2_test1_level2; + + --%test(Test 2 from Suite2_2 on level 2) + --%tags(level2,test2,test2_level2) + procedure suite2_2_test2_level2; + + end suite2_2_level2_pkg; + ]'; + + execute immediate q'[create or replace package body suite2_2_level2_pkg is + procedure suite2_2_test1_level2 is + begin + dbms_output.put_line('suite2_2_level2_pkg.suite2_2_test1_level2 executed'); + end; + procedure suite2_2_test2_level2 is + begin + dbms_output.put_line('suite2_2_level2_pkg.suite2_2_test2_level2 executed'); + end; + end suite2_2_level2_pkg; + ]'; + + execute immediate q'[grant execute on test_package_1 to public]'; execute immediate q'[grant execute on test_package_2 to public]'; execute immediate q'[grant execute on test_package_3 to public]'; execute immediate q'[grant execute on test_tag_pkg_1 to public]'; execute immediate q'[grant execute on test_tag_pkg_2 to public]'; - execute immediate q'[grant execute on test_tag_pkg_3 to public]'; + execute immediate q'[grant execute on test_tag_pkg_3 to public]'; + + execute immediate q'[grant execute on suite1_level1_pkg to public]'; + execute immediate q'[grant execute on suite1_1_level2_pkg to public]'; + execute immediate q'[grant execute on suite1_2_level2_pkg to public]'; + execute immediate q'[grant execute on suite2_level1_pkg to public]'; + execute immediate q'[grant execute on suite2_2_level2_pkg to public]'; end; procedure drop_ut3_user_tests is @@ -447,7 +597,13 @@ create or replace package body run_helper is execute immediate q'[drop package test_package_3]'; execute immediate q'[drop package test_tag_pkg_1]'; execute immediate q'[drop package test_tag_pkg_2]'; - execute immediate q'[drop package test_tag_pkg_3]'; + execute immediate q'[drop package test_tag_pkg_3]'; + + execute immediate q'[drop package suite2_2_level2_pkg]'; + execute immediate q'[drop package suite2_level1_pkg]'; + execute immediate q'[drop package suite1_2_level2_pkg]'; + execute immediate q'[drop package suite1_level1_pkg]'; + execute immediate q'[drop package suite1_1_level2_pkg]'; end; procedure create_test_suite is diff --git a/test/ut3_user/api/test_ut_run.pkb b/test/ut3_user/api/test_ut_run.pkb index 25b50343e..e80235744 100644 --- a/test/ut3_user/api/test_ut_run.pkb +++ b/test/ut3_user/api/test_ut_run.pkb @@ -1244,6 +1244,31 @@ procedure tag_exclude_run_fun_pth_lst_lg is ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like( '%test_package_2.test3%executed%' ); ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like( '%test_package_2.test4%executed%' ); ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like( '%test_package_3.test6%executed%' ); + + l_results := ut3_tester_helper.run_helper.run(a_tags => 'any'); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_be_like( '%suite2_level1_pkg.test1_level1 executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_be_like( '%suite2_level1_pkg.test2_level1 executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_be_like( '%suite1_level1_pkg.test1_level1 executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_be_like( '%suite2_2_level2_pkg.suite2_2_test1_level2 executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_be_like( '%suite2_2_level2_pkg.suite2_2_test2_level2 executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_be_like( '%suite1_2_level2_pkg.suite1_2_test1_level2 executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_be_like( '%suite1_2_level2_pkg.suite1_2_test2_level1 executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_be_like( '%suite1_1_level2_pkg.suite1_1_test1_level2 executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like( '%suite1_level1_pkg.test2_level1 executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like( '%suite1_1_level2_pkg.suite1_1_test2_level2 executed%' ); + + l_results := ut3_tester_helper.run_helper.run(a_tags => 'none'); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_be_like( '%suite1_level1_pkg.test2_level1 executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).to_be_like( '%suite1_1_level2_pkg.suite1_1_test2_level2 executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like( '%suite2_level1_pkg.test1_level1 executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like( '%suite2_level1_pkg.test2_level1 executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like( '%suite1_level1_pkg.test1_level1 executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like( '%suite2_2_level2_pkg.suite2_2_test1_level2 executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like( '%suite2_2_level2_pkg.suite2_2_test2_level2 executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like( '%suite1_2_level2_pkg.suite1_2_test1_level2 executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like( '%suite1_2_level2_pkg.suite1_2_test2_level1 executed%' ); + ut.expect( ut3_tester_helper.main_helper.table_to_clob(l_results) ).not_to_be_like( '%suite1_1_level2_pkg.suite1_1_test1_level2 executed%' ); + end; From 9dee7e0ffd4bfee0af948befd4f6b739506c11d1 Mon Sep 17 00:00:00 2001 From: Lukasz Wasylow Date: Tue, 25 Apr 2023 17:08:42 -0700 Subject: [PATCH 54/77] Update docs --- docs/userguide/running-unit-tests.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/userguide/running-unit-tests.md b/docs/userguide/running-unit-tests.md index b3a974d53..044ffdb35 100644 --- a/docs/userguide/running-unit-tests.md +++ b/docs/userguide/running-unit-tests.md @@ -325,7 +325,7 @@ Multiple tags are separated by comma. Tag expressions are boolean expressions created by combining tags with the `!`, `&`, `|` operators. Tag expressions can be grouped using `(` and `)` braces. Grouping tag expressions affects operator precedence. -Two special expressions are supported, `any` and `none`, which select all tests with any tags at all, and all tests without any tags, respectively. These special expressions may be combined with other expressions just like normal tags. When using `none` be aware that if the suite is tagged it will exclude any tests and children belonging to that suite. +Two special expressions are supported, `any` and `none`, which select all tests with any tags at all, and all tests without any tags, respectively. These special expressions may be combined with other expressions just like normal tags. When using `none` be aware that if the suite is tagged it will exclude any tests and suites below. | Operator | Meaning | | -------- | --------| From beb9a3a3207e208b7e5e2687b6d4751b4ee705b9 Mon Sep 17 00:00:00 2001 From: Lukasz Wasylow Date: Wed, 26 Apr 2023 17:30:32 -0700 Subject: [PATCH 55/77] Resolving PR --- docs/userguide/annotations.md | 2 +- docs/userguide/running-unit-tests.md | 8 +++++++- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/docs/userguide/annotations.md b/docs/userguide/annotations.md index 4ec6de59d..7c7015872 100644 --- a/docs/userguide/annotations.md +++ b/docs/userguide/annotations.md @@ -1634,7 +1634,7 @@ Tags must follow the below naming convention: - tag cannot start with a dash, e.g. `-some-stuff` is **not** a valid tag - tag cannot contain spaces, e.g. `test of batch`. To create a multi-word tag use underscores or dashes, e.g. `test_of_batch`, `test-of-batch` - leading and trailing spaces are ignored in tag name, e.g. `--%tags( tag1 , tag2 )` becomes `tag1` and `tag2` tag names -- tag cannot be one of two reserved words : `none` and `any`, any tags with that will not be considered. +- tag cannot be one of two reserved words: `none` and `any`. `none` and `any` as a tag will be treated as no tag ### Suitepath diff --git a/docs/userguide/running-unit-tests.md b/docs/userguide/running-unit-tests.md index 044ffdb35..e12fd6b32 100644 --- a/docs/userguide/running-unit-tests.md +++ b/docs/userguide/running-unit-tests.md @@ -325,7 +325,13 @@ Multiple tags are separated by comma. Tag expressions are boolean expressions created by combining tags with the `!`, `&`, `|` operators. Tag expressions can be grouped using `(` and `)` braces. Grouping tag expressions affects operator precedence. -Two special expressions are supported, `any` and `none`, which select all tests with any tags at all, and all tests without any tags, respectively. These special expressions may be combined with other expressions just like normal tags. When using `none` be aware that if the suite is tagged it will exclude any tests and suites below. +Two reserved keywords, `any` and `none`, can be used when creating a tag expression to run tests. +- `any` keyword represents tests and suites with any tags +- `none` keyword represents tests and suites without tags + +These keywords may be combined with other expressions just like normal tags. + +**Note:** When specifying `none`, be aware that it will exclude any tests/suites/contexts contained within a tagged suite. | Operator | Meaning | | -------- | --------| From 46ffe739386b92dc831a46227c17326bb85f459e Mon Sep 17 00:00:00 2001 From: Lukasz Wasylow Date: Thu, 27 Apr 2023 11:48:26 -0700 Subject: [PATCH 56/77] Update note --- docs/userguide/running-unit-tests.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/userguide/running-unit-tests.md b/docs/userguide/running-unit-tests.md index e12fd6b32..2b18a3d3b 100644 --- a/docs/userguide/running-unit-tests.md +++ b/docs/userguide/running-unit-tests.md @@ -331,7 +331,8 @@ Two reserved keywords, `any` and `none`, can be used when creating a tag express These keywords may be combined with other expressions just like normal tags. -**Note:** When specifying `none`, be aware that it will exclude any tests/suites/contexts contained within a tagged suite. +!!! note + When specifying `none`, be aware that it will exclude any tests/suites/contexts contained within a tagged suite. | Operator | Meaning | | -------- | --------| From 73e9cf32bce4f0b8dbd38f7da63ad253b0ae42b6 Mon Sep 17 00:00:00 2001 From: Jacek Gebal Date: Thu, 27 Apr 2023 22:09:18 +0300 Subject: [PATCH 57/77] Added missing test for multiple expectation failures with exception thrown. --- test/ut3_user/reporters.pkb | 5 +-- .../reporters/test_teamcity_reporter.pkb | 37 ++++++++++++++++++- .../reporters/test_teamcity_reporter.pks | 5 ++- 3 files changed, 41 insertions(+), 6 deletions(-) diff --git a/test/ut3_user/reporters.pkb b/test/ut3_user/reporters.pkb index b6d43b144..031683615 100644 --- a/test/ut3_user/reporters.pkb +++ b/test/ut3_user/reporters.pkb @@ -90,11 +90,10 @@ as procedure erroring_test is - l_variable integer; + l_integer_variable integer; begin dbms_output.put_line(''); - l_variable := 'a string'; - ut3_develop.ut.expect(l_variable).to_equal(1); + l_integer_variable := 'a string'; end; procedure disabled_test diff --git a/test/ut3_user/reporters/test_teamcity_reporter.pkb b/test/ut3_user/reporters/test_teamcity_reporter.pkb index 7c4c1e513..517ddc037 100644 --- a/test/ut3_user/reporters/test_teamcity_reporter.pkb +++ b/test/ut3_user/reporters/test_teamcity_reporter.pkb @@ -36,6 +36,9 @@ create or replace package body test_teamcity_reporter as --%test procedure multi_failure; + + --%test + procedure multi_failure_on_error; end;]'; execute immediate q'[create or replace package body check_multiple_failures is procedure multi_failure is @@ -44,6 +47,14 @@ create or replace package body test_teamcity_reporter as ut3_develop.ut.expect(2).to_equal(1); ut3_develop.ut.expect('Bad').to_equal('Good'); end; + procedure multi_failure_on_error is + l_integer_variable integer; + begin + ut3_develop.ut.expect(1).to_be_null; + ut3_develop.ut.expect(2).to_equal(1); + ut3_develop.ut.expect('Bad').to_equal('Good'); + l_integer_variable := 'a string'; + end; end;]'; end; @@ -132,7 +143,7 @@ create or replace package body test_teamcity_reporter as ut.expect(ut3_tester_helper.main_helper.table_to_clob(l_output_data)).to_be_like(l_expected); end; - procedure report_mutiple_expectations is + procedure report_multiple_expectations is l_output_data ut3_develop.ut_varchar2_list; l_expected varchar2(32767); begin @@ -146,7 +157,29 @@ create or replace package body test_teamcity_reporter as --act select * bulk collect into l_output_data - from table(ut3_develop.ut.run('check_multiple_failures',ut3_develop.ut_teamcity_reporter())); + from table(ut3_develop.ut.run('check_multiple_failures.multi_failure',ut3_develop.ut_teamcity_reporter())); + + --assert + ut.expect(ut3_tester_helper.main_helper.table_to_clob(l_output_data)).to_be_like(l_expected); + end; + + procedure report_multiple_expect_on_err is + l_output_data ut3_develop.ut_varchar2_list; + l_expected varchar2(32767); + begin + l_expected := q'{%##teamcity[testSuiteStarted timestamp='%' name='check_multiple_failures'] +%##teamcity[testStarted timestamp='%' captureStandardOutput='true' name='ut3_user.check_multiple_failures.multi_failure_on_error'] +%##teamcity[testStdErr timestamp='%' name='ut3_user.check_multiple_failures.multi_failure_on_error' out='Test exception:|nORA-06502: PL/SQL: %: character to number conversion error|nORA-06512: at "UT3_USER.CHECK_MULTIPLE_FAILURES", line %|nORA-06512: at %|n'] +%##teamcity[testFailed timestamp='%' details='Test exception:|nORA-06502: PL/SQL: %: character to number conversion error|nORA-06512: at "UT3_USER.CHECK_MULTIPLE_FAILURES", line %|nORA-06512: at %|n' message='Error occured' name='ut3_user.check_multiple_failures.multi_failure_on_error'] +%##teamcity[testFailed timestamp='%' details='Actual: 1 (number) was expected to be null' name='ut3_user.check_multiple_failures.multi_failure_on_error'] +%##teamcity[testFailed timestamp='%' details='Actual: 2 (number) was expected to equal: 1 (number)' name='ut3_user.check_multiple_failures.multi_failure_on_error'] +%##teamcity[testFailed timestamp='%' details='Actual: |'Bad|' (varchar2) was expected to equal: |'Good|' (varchar2)' name='ut3_user.check_multiple_failures.multi_failure_on_error'] +%##teamcity[testFinished timestamp='%' duration='%' name='ut3_user.check_multiple_failures.multi_failure_on_error'] +%##teamcity[testSuiteFinished timestamp='%' name='check_multiple_failures']}'; + --act + select * + bulk collect into l_output_data + from table(ut3_develop.ut.run('check_multiple_failures.multi_failure_on_error',ut3_develop.ut_teamcity_reporter())); --assert ut.expect(ut3_tester_helper.main_helper.table_to_clob(l_output_data)).to_be_like(l_expected); diff --git a/test/ut3_user/reporters/test_teamcity_reporter.pks b/test/ut3_user/reporters/test_teamcity_reporter.pks index 30ae7e26b..1b9277e7a 100644 --- a/test/ut3_user/reporters/test_teamcity_reporter.pks +++ b/test/ut3_user/reporters/test_teamcity_reporter.pks @@ -16,7 +16,10 @@ create or replace package test_teamcity_reporter as procedure trims_long_output; --%test(Reports failures on multiple expectations) - procedure report_mutiple_expectations; + procedure report_multiple_expectations; + + --%test(Reports failures on multiple expectations) + procedure report_multiple_expect_on_err; --%afterall procedure remove_test_package; From 6a32cbd572dd9cbbcb8858a32693cd6f41376267 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" Date: Fri, 28 Apr 2023 10:29:06 +0100 Subject: [PATCH 58/77] Updated project version after build [skip ci] --- docs/about/authors.md | 2 +- docs/about/license.md | 2 +- docs/about/project-details.md | 2 +- docs/about/support.md | 2 +- docs/index.md | 2 +- docs/userguide/advanced_data_comparison.md | 2 +- docs/userguide/annotations.md | 2 +- docs/userguide/best-practices.md | 2 +- docs/userguide/coverage.md | 2 +- docs/userguide/exception-reporting.md | 2 +- docs/userguide/expectations.md | 2 +- docs/userguide/getting-started.md | 2 +- docs/userguide/install.md | 2 +- docs/userguide/querying_suites.md | 2 +- docs/userguide/reporters.md | 2 +- docs/userguide/running-unit-tests.md | 2 +- docs/userguide/upgrade.md | 2 +- source/core/ut_utils.pks | 2 +- 18 files changed, 18 insertions(+), 18 deletions(-) diff --git a/docs/about/authors.md b/docs/about/authors.md index a47004a3c..afdf64bdd 100644 --- a/docs/about/authors.md +++ b/docs/about/authors.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4105--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4167--develop-blue.svg) ### utPLSQL v3 Major Contributors diff --git a/docs/about/license.md b/docs/about/license.md index 8b6437299..37bc28342 100644 --- a/docs/about/license.md +++ b/docs/about/license.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4105--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4167--develop-blue.svg) # Version Information diff --git a/docs/about/project-details.md b/docs/about/project-details.md index 02eabab07..c4a36c2fc 100644 --- a/docs/about/project-details.md +++ b/docs/about/project-details.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4105--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4167--develop-blue.svg) # utPLSQL Project Details diff --git a/docs/about/support.md b/docs/about/support.md index aacb289f2..450bde3ad 100644 --- a/docs/about/support.md +++ b/docs/about/support.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4105--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4167--develop-blue.svg) # How to get support diff --git a/docs/index.md b/docs/index.md index 5e30a9fc6..926b89912 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4105--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4167--develop-blue.svg) ## What is utPLSQL diff --git a/docs/userguide/advanced_data_comparison.md b/docs/userguide/advanced_data_comparison.md index 8db920ee7..2efea8de6 100644 --- a/docs/userguide/advanced_data_comparison.md +++ b/docs/userguide/advanced_data_comparison.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4105--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4167--develop-blue.svg) # Advanced data comparison diff --git a/docs/userguide/annotations.md b/docs/userguide/annotations.md index 7c7015872..0ef65a9e8 100644 --- a/docs/userguide/annotations.md +++ b/docs/userguide/annotations.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4105--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4167--develop-blue.svg) Annotations are used to configure tests and suites in a declarative way similar to modern OOP languages. This way, test configuration is stored along with the test logic inside the test package. No additional configuration files or tables are needed for test cases. The annotation names are based on popular testing frameworks such as JUnit. diff --git a/docs/userguide/best-practices.md b/docs/userguide/best-practices.md index b83fb78c3..3582c1968 100644 --- a/docs/userguide/best-practices.md +++ b/docs/userguide/best-practices.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4105--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4167--develop-blue.svg) The following are best practices we at utPLSQL have learned about PL/SQL and Unit Testing. diff --git a/docs/userguide/coverage.md b/docs/userguide/coverage.md index 04b3db380..35cdd3a3f 100644 --- a/docs/userguide/coverage.md +++ b/docs/userguide/coverage.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4105--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4167--develop-blue.svg) utPLSQL comes with a built-in coverage reporting engine. The code coverage reporting uses package DBMS_PROFILER (and DBMS_PLSQL_CODE_COVERAGE on Oracle database version 12.2 and above) provided with Oracle database. Code coverage is gathered for the following source types: diff --git a/docs/userguide/exception-reporting.md b/docs/userguide/exception-reporting.md index a7b0ae0a5..bbda24125 100644 --- a/docs/userguide/exception-reporting.md +++ b/docs/userguide/exception-reporting.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4105--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4167--develop-blue.svg) utPLSQL is responsible for handling exceptions wherever they occur in the test run. The framework is trapping most of the exceptions so that the test execution is not affected by individual tests or test packages throwing an exception. The framework provides a full stacktrace for every exception that was thrown. The reported stacktrace does not include any utPLSQL library calls in it. diff --git a/docs/userguide/expectations.md b/docs/userguide/expectations.md index 2b436250e..af07865b1 100644 --- a/docs/userguide/expectations.md +++ b/docs/userguide/expectations.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4105--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4167--develop-blue.svg) ## Expectation concepts diff --git a/docs/userguide/getting-started.md b/docs/userguide/getting-started.md index 37c22c882..afd4ad124 100644 --- a/docs/userguide/getting-started.md +++ b/docs/userguide/getting-started.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4105--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4167--develop-blue.svg) # Getting started with TDD and utPLSQL diff --git a/docs/userguide/install.md b/docs/userguide/install.md index 00a74b288..7e6af19cd 100644 --- a/docs/userguide/install.md +++ b/docs/userguide/install.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4105--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4167--develop-blue.svg) ## Supported database versions diff --git a/docs/userguide/querying_suites.md b/docs/userguide/querying_suites.md index 164c0cbc0..0a5584ba9 100644 --- a/docs/userguide/querying_suites.md +++ b/docs/userguide/querying_suites.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4105--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4167--develop-blue.svg) ## Obtaining information about suites diff --git a/docs/userguide/reporters.md b/docs/userguide/reporters.md index e92558e3d..ca62b674e 100644 --- a/docs/userguide/reporters.md +++ b/docs/userguide/reporters.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4105--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4167--develop-blue.svg) utPLSQL provides several reporting formats. The sections below describe most of them. diff --git a/docs/userguide/running-unit-tests.md b/docs/userguide/running-unit-tests.md index 2b18a3d3b..a24145f82 100644 --- a/docs/userguide/running-unit-tests.md +++ b/docs/userguide/running-unit-tests.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4105--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4167--develop-blue.svg) utPLSQL framework provides two main entry points to run unit tests from within the database: diff --git a/docs/userguide/upgrade.md b/docs/userguide/upgrade.md index 794e75070..160c51fd4 100644 --- a/docs/userguide/upgrade.md +++ b/docs/userguide/upgrade.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4105--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4167--develop-blue.svg) # Upgrading from version 2 diff --git a/source/core/ut_utils.pks b/source/core/ut_utils.pks index b60fc4cdf..954d4f8be 100644 --- a/source/core/ut_utils.pks +++ b/source/core/ut_utils.pks @@ -21,7 +21,7 @@ create or replace package ut_utils authid definer is * */ - gc_version constant varchar2(50) := 'v3.1.14.4105-develop'; + gc_version constant varchar2(50) := 'v3.1.14.4167-develop'; subtype t_executable_type is varchar2(30); gc_before_all constant t_executable_type := 'beforeall'; From 962f6fd4446d8369af56360dcc591e2a7cc919f7 Mon Sep 17 00:00:00 2001 From: Jacek Gebal Date: Sun, 21 May 2023 12:54:53 +0300 Subject: [PATCH 59/77] Resolves #1252 Replaced all regexp ranges with posix representations to avoid NLS_SORT impact on regex behavior. --- .../core/annotations/ut_annotation_manager.pkb | 2 +- .../core/annotations/ut_annotation_parser.pkb | 2 +- source/core/types/ut_executable_test.tpb | 2 +- source/core/ut_expectation_processor.pkb | 12 ++++++------ source/core/ut_metadata.pkb | 2 +- source/core/ut_suite_cache_manager.pkb | 4 ++-- source/core/ut_suite_manager.pkb | 8 ++++---- source/core/ut_utils.pkb | 6 +++--- test/ut3_tester/core/test_suite_manager.pkb | 17 +++++++++++++++++ test/ut3_tester/core/test_suite_manager.pks | 10 ++++++++++ 10 files changed, 46 insertions(+), 19 deletions(-) diff --git a/source/core/annotations/ut_annotation_manager.pkb b/source/core/annotations/ut_annotation_manager.pkb index 16923bc23..65f7b3e40 100644 --- a/source/core/annotations/ut_annotation_manager.pkb +++ b/source/core/annotations/ut_annotation_manager.pkb @@ -246,7 +246,7 @@ create or replace package body ut_annotation_manager as l_sql_clob := regexp_replace(l_sql_clob, '^(.*?\s*create(\s+or\s+replace)?(\s+(editionable|noneditionable))?\s+?)((package|type).*)', '\5', 1, 1, 'ni'); -- remove "OWNER." from create or replace statement. -- Owner is not supported along with AUTHID - see issue https://github.com/utPLSQL/utPLSQL/issues/1088 - l_sql_clob := regexp_replace(l_sql_clob, '^(package|type)\s+("?[a-zA-Z][a-zA-Z0-9#_$]*"?\.)(.*)', '\1 \3', 1, 1, 'ni'); + l_sql_clob := regexp_replace(l_sql_clob, '^(package|type)\s+("?[[:alpha:]][[:alnum:]$#_]*"?\.)(.*)', '\1 \3', 1, 1, 'ni'); l_sql_lines := ut_utils.convert_collection( ut_utils.clob_to_table(l_sql_clob) ); end if; open l_result for diff --git a/source/core/annotations/ut_annotation_parser.pkb b/source/core/annotations/ut_annotation_parser.pkb index f0271c957..10bb76b3c 100644 --- a/source/core/annotations/ut_annotation_parser.pkb +++ b/source/core/annotations/ut_annotation_parser.pkb @@ -25,7 +25,7 @@ create or replace package body ut_annotation_parser as gc_annot_comment_pattern constant varchar2(30) := '^( |'||chr(09)||')*-- *('||gc_annotation_qualifier||'.*?)$'; -- chr(09) is a tab character gc_comment_replacer_patter constant varchar2(50) := '{COMMENT#%N%}'; gc_comment_replacer_regex_ptrn constant varchar2(25) := '{COMMENT#(\d+)}'; - gc_regexp_identifier constant varchar2(50) := '[a-zA-Z][a-zA-Z0-9#_$]*'; + gc_regexp_identifier constant varchar2(50) := '[[:alpha:]][[:alnum:]$#_]*'; gc_annotation_block_pattern constant varchar2(200) := '(({COMMENT#.+}'||chr(10)||')+)( |'||chr(09)||')*(procedure|function)\s+(' || gc_regexp_identifier || ')'; gc_annotation_pattern constant varchar2(50) := gc_annotation_qualifier || gc_regexp_identifier || '[ '||chr(9)||']*(\(.*?\)\s*?$)?'; diff --git a/source/core/types/ut_executable_test.tpb b/source/core/types/ut_executable_test.tpb index c5a7f29a1..fa5872e04 100644 --- a/source/core/types/ut_executable_test.tpb +++ b/source/core/types/ut_executable_test.tpb @@ -159,7 +159,7 @@ create or replace type body ut_executable_test as if self.error_stack is null then l_fail_message := 'Expected one of exceptions ('||l_expected_error_codes||') but nothing was raised.'; else - l_actual_error_no := regexp_substr(self.error_stack, '^[a-zA-Z]{3}(-[0-9]+)', subexpression=>1); + l_actual_error_no := regexp_substr(self.error_stack, '^[[:alpha:]]{3}(-[0-9]+)', subexpression=>1); if not l_actual_error_no member of a_expected_error_codes or l_actual_error_no is null then l_fail_message := 'Actual: '||l_actual_error_no||' was expected to '; if cardinality(a_expected_error_codes) > 1 then diff --git a/source/core/ut_expectation_processor.pkb b/source/core/ut_expectation_processor.pkb index 77b734b30..c165a9ee5 100644 --- a/source/core/ut_expectation_processor.pkb +++ b/source/core/ut_expectation_processor.pkb @@ -159,17 +159,17 @@ create or replace package body ut_expectation_processor as -- when 11g and 12c reports only package name function cut_header_and_expectations( a_stack varchar2 ) return varchar2 is begin - return regexp_substr( a_stack, '(.*\.(UT_EQUAL|UT_BE_WITHIN[A-Z0-9#_$]*|UT_EXPECTATION[A-Z0-9#_$]*|UT|UTASSERT2?)(\.[A-Z0-9#_$]+)?\s+)+((.|\s)*)', 1, 1, 'm', 4); + return regexp_substr( a_stack, '(.*\.(UT_EQUAL|UT_BE_WITHIN[[:alnum:]$#_]*|UT_EXPECTATION[[:alnum:]$#_]*|UT|UTASSERT2?)(\.[[:alnum:]$#_]+)?\s+)+((.|\s)*)', 1, 1, 'm', 4); end; function cut_address_columns( a_stack varchar2 ) return varchar2 is begin - return regexp_replace( a_stack, '^(0x)?[0-9a-f]+\s+', '', 1, 0, 'mi' ); + return regexp_replace( a_stack, '^(0x)?[[:digit:]abcdef]+\s+', '', 1, 0, 'mi' ); end; function cut_framework_stack( a_stack varchar2 ) return varchar2 is begin return regexp_replace( a_stack, - '[0-9]+\s+anonymous\s+block\s+[0-9]+\s+package\s+body\s+sys\.dbms_sql(\.execute)?\s+[0-9]+\s+[0-9_$#a-z ]+\.ut_executable.*', + '[0-9]+\s+anonymous\s+block\s+[0-9]+\s+package\s+body\s+sys\.dbms_sql(\.execute)?\s+[0-9]+\s+[[:alnum:]_$# ]+\.ut_executable.*', '', 1, 1, 'mni' ); @@ -178,7 +178,7 @@ create or replace package body ut_expectation_processor as begin return regexp_replace( a_stack, - '([0-9]+)\s+(.* )?((anonymous block)|(([0-9_$#a-z]+\.[0-9_$#a-z]+(\.([0-9_$#a-z])+)?)))', + '([0-9]+)\s+(.* )?((anonymous block)|(([[:alnum:]$#_]+\.[[:alnum:]$#_]+(\.([[:alnum:]$#_])+)?)))', 'at "\3", line \1', 1, 0, 'i' ); end; @@ -190,8 +190,8 @@ create or replace package body ut_expectation_processor as l_caller_stack_line := regexp_substr(l_call_stack,'^(.*)'); if l_caller_stack_line like '%.%' then l_line_no := to_number( regexp_substr( l_caller_stack_line, ', line (\d+)', subexpression => 1 ) ); - l_owner := regexp_substr( l_caller_stack_line, 'at "([A-Za-z0-9$#_]+)\.(([A-Za-z0-9$#_]+)(\.([A-Za-z0-9$#_]+))?)", line (\d+)', subexpression => 1 ); - l_object_name := regexp_substr( l_caller_stack_line, 'at "([A-Za-z0-9$#_]+)\.(([A-Za-z0-9$#_]+)(\.([A-Za-z0-9$#_]+))?)", line (\d+)', subexpression => 3 ); + l_owner := regexp_substr( l_caller_stack_line, 'at "([[:alnum:]$#_]+)\.(([[:alnum:]$#_]+)(\.([[:alnum:]$#_]+))?)", line (\d+)', subexpression => 1 ); + l_object_name := regexp_substr( l_caller_stack_line, 'at "([[:alnum:]$#_]+)\.(([[:alnum:]$#_]+)(\.([[:alnum:]$#_]+))?)", line (\d+)', subexpression => 3 ); l_result := l_caller_stack_line || ' ' || rtrim(ut_metadata.get_source_definition_line(l_owner, l_object_name, l_line_no),chr(10)) || replace( l_call_stack, l_caller_stack_line ); diff --git a/source/core/ut_metadata.pkb b/source/core/ut_metadata.pkb index 700c0efb0..360b7b97d 100644 --- a/source/core/ut_metadata.pkb +++ b/source/core/ut_metadata.pkb @@ -304,7 +304,7 @@ create or replace package body ut_metadata as begin l_result := regexp_substr( a_full_object_name, - '^([A-Za-z0-9$#_]+|".*?")\.([A-Za-z0-9$#_]+|".*?")', subexpression => 2 + '^([[:alnum:]$#_]+|".*?")\.([[:alnum:]$#_]+|".*?")', subexpression => 2 ); if not l_result like '"%"' then l_result := upper(l_result); diff --git a/source/core/ut_suite_cache_manager.pkb b/source/core/ut_suite_cache_manager.pkb index 680624f02..6d621c339 100644 --- a/source/core/ut_suite_cache_manager.pkb +++ b/source/core/ut_suite_cache_manager.pkb @@ -88,7 +88,7 @@ create or replace package body ut_suite_cache_manager is end; function group_paths_by_schema(a_paths ut_varchar2_list) return ut_path_items is - c_package_path_regex constant varchar2(100) := '^([A-Za-z0-9$#_]+)(\.([A-Za-z0-9$#_\*]+))?(\.([A-Za-z0-9$#_\*]+))?$'; + c_package_path_regex constant varchar2(100) := '^([[:alnum:]$#_]+)(\.([[:alnum:]$#_\*]+))?(\.([[:alnum:]$#_\*]+))?$'; l_results ut_path_items := ut_path_items(); l_path_item ut_path_item; i pls_integer; @@ -157,7 +157,7 @@ create or replace package body ut_suite_cache_manager is from schema_paths sp left outer join ut_suite_cache c on ( c.object_owner = upper(sp.schema_name) --and c.path like replace(sp.suite_path,'*','%')) - and regexp_like(c.path,'^'||replace(sp.suite_path,'*','[A-Za-z0-9$#_]*'))) + and regexp_like(c.path,'^'||replace(sp.suite_path,'*','[[:alnum:]$#_]*'))) where sp.suite_path is not null and instr(sp.suite_path,'*') > 0 ), straigth_suite_paths as ( diff --git a/source/core/ut_suite_manager.pkb b/source/core/ut_suite_manager.pkb index c46ba66b8..c418de638 100644 --- a/source/core/ut_suite_manager.pkb +++ b/source/core/ut_suite_manager.pkb @@ -32,7 +32,7 @@ create or replace package body ut_suite_manager is for i in 1 .. a_paths.count loop l_path := a_paths(i); if l_path is null or not ( - regexp_like(l_path, '^[A-Za-z0-9$#_\*]+(\.[A-Za-z0-9$#_\*]+){0,2}$') or regexp_like(l_path, '^([A-Za-z0-9$#_]+)?:[A-Za-z0-9$#_\*]+(\.[A-Za-z0-9$#_\*]+)*$')) then + regexp_like(l_path, '^[[:alnum:]$#_\*]+(\.[[:alnum:]$#_\*]+){0,2}$') or regexp_like(l_path, '^([[:alnum:]$#_]+)?:[[:alnum:]$#_\*]+(\.[[:alnum:]$#_\*]+)*$')) then raise_application_error(ut_utils.gc_invalid_path_format, 'Invalid path format: ' || nvl(l_path, 'NULL')); end if; end loop; @@ -61,9 +61,9 @@ create or replace package body ut_suite_manager is for i in 1 .. a_paths.count loop --if path is suite-path - if regexp_like(a_paths(i), '^([A-Za-z0-9$#_]+)?:') then + if regexp_like(a_paths(i), '^([[:alnum:]$#_]+)?:') then --get schema name / path - l_schema := regexp_substr(a_paths(i), '^([A-Za-z0-9$#_]+)?:',subexpression => 1); + l_schema := regexp_substr(a_paths(i), '^([[:alnum:]$#_]+)?:',subexpression => 1); -- transform ":path1[.path2]" to "schema:path1[.path2]" if l_schema is not null then l_schema := sys.dbms_assert.schema_name(upper(l_schema)); @@ -78,7 +78,7 @@ create or replace package body ut_suite_manager is -- Object name or procedure is allowed to have filter char -- However this is not allowed on schema begin - l_object := regexp_substr(a_paths(i), '^[A-Za-z0-9$#_\*]+'); + l_object := regexp_substr(a_paths(i), '^[[:alnum:]$#_\*]+'); l_schema := sys.dbms_assert.schema_name(upper(l_object)); exception when sys.dbms_assert.invalid_schema_name then diff --git a/source/core/ut_utils.pkb b/source/core/ut_utils.pkb index 6f972e11c..f29e3b188 100644 --- a/source/core/ut_utils.pkb +++ b/source/core/ut_utils.pkb @@ -19,9 +19,9 @@ create or replace package body ut_utils is /** * Constants regex used to validate XML name */ - gc_invalid_first_xml_char constant varchar2(50) := '[^_a-zA-Z]'; - gc_invalid_xml_char constant varchar2(50) := '[^_a-zA-Z0-9\.-]'; - gc_full_valid_xml_name constant varchar2(50) := '^([_a-zA-Z])([_a-zA-Z0-9\.-])*$'; + gc_invalid_first_xml_char constant varchar2(50) := '[^_[:alpha:]]'; + gc_invalid_xml_char constant varchar2(50) := '[^_[:alnum:]\.-]'; + gc_full_valid_xml_name constant varchar2(50) := '^([[:alpha:]])([_[:alnum:]\.-])*$'; gc_owner_hash constant integer(11) := dbms_utility.get_hash_value( ut_owner(), 0, power(2,31)-1); diff --git a/test/ut3_tester/core/test_suite_manager.pkb b/test/ut3_tester/core/test_suite_manager.pkb index fbac0f3f5..c818ad212 100644 --- a/test/ut3_tester/core/test_suite_manager.pkb +++ b/test/ut3_tester/core/test_suite_manager.pkb @@ -2217,5 +2217,22 @@ end;]'; end loop; end; + --%test(Path validation does not fail on Estonian NLS_SORT - fix #1252) + procedure path_validate_nls_sort is + l_schema_names ut3_develop.ut_varchar2_rows; + begin + --Arrange + execute immediate q'[alter session set nls_sort='estonian']'; + --Act + l_schema_names := ut3_develop.ut_suite_manager.get_schema_names(ut3_develop.ut_varchar2_list('ut3')); + --Asseert + ut.expect(sqlcode).to_equal(0); + end; + + + procedure reset_nls_sort is + begin + execute immediate q'[alter session set nls_sort='binary']'; + end; end test_suite_manager; / diff --git a/test/ut3_tester/core/test_suite_manager.pks b/test/ut3_tester/core/test_suite_manager.pks index d9d4efc09..ac7818f4f 100644 --- a/test/ut3_tester/core/test_suite_manager.pks +++ b/test/ut3_tester/core/test_suite_manager.pks @@ -237,5 +237,15 @@ create or replace package test_suite_manager is --%endcontext + --%context(paths validation) + + --%test(Path validation does not fail on Estonian NLS_SORT - fix #1252) + --%aftertest(reset_nls_sort) + procedure path_validate_nls_sort; + + procedure reset_nls_sort; + + --%endcontext + end test_suite_manager; / From 6589b8468ecaaf85258873e6484c0183b770dc17 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" Date: Sat, 27 May 2023 12:42:57 +0100 Subject: [PATCH 60/77] Updated project version after build [skip ci] --- docs/about/authors.md | 2 +- docs/about/license.md | 2 +- docs/about/project-details.md | 2 +- docs/about/support.md | 2 +- docs/index.md | 2 +- docs/userguide/advanced_data_comparison.md | 2 +- docs/userguide/annotations.md | 2 +- docs/userguide/best-practices.md | 2 +- docs/userguide/coverage.md | 2 +- docs/userguide/exception-reporting.md | 2 +- docs/userguide/expectations.md | 2 +- docs/userguide/getting-started.md | 2 +- docs/userguide/install.md | 2 +- docs/userguide/querying_suites.md | 2 +- docs/userguide/reporters.md | 2 +- docs/userguide/running-unit-tests.md | 2 +- docs/userguide/upgrade.md | 2 +- source/core/ut_utils.pks | 2 +- 18 files changed, 18 insertions(+), 18 deletions(-) diff --git a/docs/about/authors.md b/docs/about/authors.md index afdf64bdd..9d18766ae 100644 --- a/docs/about/authors.md +++ b/docs/about/authors.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4167--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4171--develop-blue.svg) ### utPLSQL v3 Major Contributors diff --git a/docs/about/license.md b/docs/about/license.md index 37bc28342..a9cd17bc9 100644 --- a/docs/about/license.md +++ b/docs/about/license.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4167--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4171--develop-blue.svg) # Version Information diff --git a/docs/about/project-details.md b/docs/about/project-details.md index c4a36c2fc..0b1b005c6 100644 --- a/docs/about/project-details.md +++ b/docs/about/project-details.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4167--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4171--develop-blue.svg) # utPLSQL Project Details diff --git a/docs/about/support.md b/docs/about/support.md index 450bde3ad..639dc2970 100644 --- a/docs/about/support.md +++ b/docs/about/support.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4167--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4171--develop-blue.svg) # How to get support diff --git a/docs/index.md b/docs/index.md index 926b89912..bd908d60c 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4167--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4171--develop-blue.svg) ## What is utPLSQL diff --git a/docs/userguide/advanced_data_comparison.md b/docs/userguide/advanced_data_comparison.md index 2efea8de6..eb3b311a5 100644 --- a/docs/userguide/advanced_data_comparison.md +++ b/docs/userguide/advanced_data_comparison.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4167--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4171--develop-blue.svg) # Advanced data comparison diff --git a/docs/userguide/annotations.md b/docs/userguide/annotations.md index 0ef65a9e8..4f1c6b420 100644 --- a/docs/userguide/annotations.md +++ b/docs/userguide/annotations.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4167--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4171--develop-blue.svg) Annotations are used to configure tests and suites in a declarative way similar to modern OOP languages. This way, test configuration is stored along with the test logic inside the test package. No additional configuration files or tables are needed for test cases. The annotation names are based on popular testing frameworks such as JUnit. diff --git a/docs/userguide/best-practices.md b/docs/userguide/best-practices.md index 3582c1968..f5f889ca6 100644 --- a/docs/userguide/best-practices.md +++ b/docs/userguide/best-practices.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4167--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4171--develop-blue.svg) The following are best practices we at utPLSQL have learned about PL/SQL and Unit Testing. diff --git a/docs/userguide/coverage.md b/docs/userguide/coverage.md index 35cdd3a3f..dbd584a7c 100644 --- a/docs/userguide/coverage.md +++ b/docs/userguide/coverage.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4167--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4171--develop-blue.svg) utPLSQL comes with a built-in coverage reporting engine. The code coverage reporting uses package DBMS_PROFILER (and DBMS_PLSQL_CODE_COVERAGE on Oracle database version 12.2 and above) provided with Oracle database. Code coverage is gathered for the following source types: diff --git a/docs/userguide/exception-reporting.md b/docs/userguide/exception-reporting.md index bbda24125..cad56f1b8 100644 --- a/docs/userguide/exception-reporting.md +++ b/docs/userguide/exception-reporting.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4167--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4171--develop-blue.svg) utPLSQL is responsible for handling exceptions wherever they occur in the test run. The framework is trapping most of the exceptions so that the test execution is not affected by individual tests or test packages throwing an exception. The framework provides a full stacktrace for every exception that was thrown. The reported stacktrace does not include any utPLSQL library calls in it. diff --git a/docs/userguide/expectations.md b/docs/userguide/expectations.md index af07865b1..67f851f36 100644 --- a/docs/userguide/expectations.md +++ b/docs/userguide/expectations.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4167--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4171--develop-blue.svg) ## Expectation concepts diff --git a/docs/userguide/getting-started.md b/docs/userguide/getting-started.md index afd4ad124..b2c67054e 100644 --- a/docs/userguide/getting-started.md +++ b/docs/userguide/getting-started.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4167--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4171--develop-blue.svg) # Getting started with TDD and utPLSQL diff --git a/docs/userguide/install.md b/docs/userguide/install.md index 7e6af19cd..4e1b3e903 100644 --- a/docs/userguide/install.md +++ b/docs/userguide/install.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4167--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4171--develop-blue.svg) ## Supported database versions diff --git a/docs/userguide/querying_suites.md b/docs/userguide/querying_suites.md index 0a5584ba9..d68240a5d 100644 --- a/docs/userguide/querying_suites.md +++ b/docs/userguide/querying_suites.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4167--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4171--develop-blue.svg) ## Obtaining information about suites diff --git a/docs/userguide/reporters.md b/docs/userguide/reporters.md index ca62b674e..ca3878f9f 100644 --- a/docs/userguide/reporters.md +++ b/docs/userguide/reporters.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4167--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4171--develop-blue.svg) utPLSQL provides several reporting formats. The sections below describe most of them. diff --git a/docs/userguide/running-unit-tests.md b/docs/userguide/running-unit-tests.md index a24145f82..4fdb74073 100644 --- a/docs/userguide/running-unit-tests.md +++ b/docs/userguide/running-unit-tests.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4167--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4171--develop-blue.svg) utPLSQL framework provides two main entry points to run unit tests from within the database: diff --git a/docs/userguide/upgrade.md b/docs/userguide/upgrade.md index 160c51fd4..bd401f5d9 100644 --- a/docs/userguide/upgrade.md +++ b/docs/userguide/upgrade.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4167--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4171--develop-blue.svg) # Upgrading from version 2 diff --git a/source/core/ut_utils.pks b/source/core/ut_utils.pks index 954d4f8be..90c165bbb 100644 --- a/source/core/ut_utils.pks +++ b/source/core/ut_utils.pks @@ -21,7 +21,7 @@ create or replace package ut_utils authid definer is * */ - gc_version constant varchar2(50) := 'v3.1.14.4167-develop'; + gc_version constant varchar2(50) := 'v3.1.14.4171-develop'; subtype t_executable_type is varchar2(30); gc_before_all constant t_executable_type := 'beforeall'; From 8b45b0d670d7212f4e1ffbc9635a4a1b2a621cf1 Mon Sep 17 00:00:00 2001 From: Jacek Gebal Date: Sun, 28 May 2023 16:00:15 +0300 Subject: [PATCH 61/77] Fix output length error and output buffer. When dealing with multibyte strings, the substring limit of 4000 **chars** is not correct. We need to apply a limit of 4000 **bytes** to both VARCHAR2 and to CLOB data types. This means we need to have a function to calculate lenght of CLOB in bytes. This is now added and all the conversions/trimming/string-splitting is done using the string length in **bytes** Additionally, a different approach (with sequences) was introduced into output buffers to avoid the sometime occurring PK violation on the output buffer tables. The new approach may add a but overhead on reading from / saving into buffer, but will definitely address the threat of PK errors. Resolves #1254 Resolves #1128 Replaced all regexp ranges with posix representations to avoid NLS_SORT impact on regex behavior. --- .../output_buffers/ut_output_buffer_base.tpb | 1 - .../output_buffers/ut_output_buffer_base.tps | 1 - .../output_buffers/ut_output_buffer_tmp.sql | 1 + .../output_buffers/ut_output_bulk_buffer.tpb | 9 ++-- .../ut_output_clob_buffer_tmp.sql | 2 + .../ut_output_clob_table_buffer.tpb | 17 ++----- .../output_buffers/ut_output_table_buffer.tpb | 27 ++++------ source/core/ut_utils.pkb | 48 ++++++++++++++--- source/core/ut_utils.pks | 5 ++ source/uninstall_objects.sql | 4 ++ test/ut3_tester/core/test_output_buffer.pkb | 51 ++++++++++++++++--- test/ut3_tester/core/test_output_buffer.pks | 6 +++ test/ut3_tester/core/test_ut_utils.pkb | 18 +++++++ test/ut3_tester/core/test_ut_utils.pks | 13 +++-- 14 files changed, 149 insertions(+), 54 deletions(-) diff --git a/source/core/output_buffers/ut_output_buffer_base.tpb b/source/core/output_buffers/ut_output_buffer_base.tpb index be4c92bd8..20cec335e 100644 --- a/source/core/output_buffers/ut_output_buffer_base.tpb +++ b/source/core/output_buffers/ut_output_buffer_base.tpb @@ -24,7 +24,6 @@ create or replace type body ut_output_buffer_base is self.self_type := coalesce(a_self_type,self.self_type); self.output_id := coalesce(a_output_id, self.output_id, sys_guid()); self.start_date := coalesce(self.start_date, sysdate); - self.last_write_message_id := 0; select /*+ no_parallel */ count(*) into l_exists from ut_output_buffer_info_tmp where output_id = self.output_id; if ( l_exists > 0 ) then update /*+ no_parallel */ ut_output_buffer_info_tmp set start_date = self.start_date where output_id = self.output_id; diff --git a/source/core/output_buffers/ut_output_buffer_base.tps b/source/core/output_buffers/ut_output_buffer_base.tps index 53be365ae..f2d9ec686 100644 --- a/source/core/output_buffers/ut_output_buffer_base.tps +++ b/source/core/output_buffers/ut_output_buffer_base.tps @@ -19,7 +19,6 @@ create or replace type ut_output_buffer_base force authid definer as object( output_id raw(32), is_closed number(1,0), start_date date, - last_write_message_id number(38,0), lock_handle varchar2(30 byte), self_type varchar2(250 byte), member procedure init(self in out nocopy ut_output_buffer_base, a_output_id raw := null, a_self_type varchar2 := null), diff --git a/source/core/output_buffers/ut_output_buffer_tmp.sql b/source/core/output_buffers/ut_output_buffer_tmp.sql index 57027817e..1ab19e534 100644 --- a/source/core/output_buffers/ut_output_buffer_tmp.sql +++ b/source/core/output_buffers/ut_output_buffer_tmp.sql @@ -30,3 +30,4 @@ create table ut_output_buffer_tmp( ) organization index nologging initrans 100 overflow nologging initrans 100; +create sequence ut_output_buffer_tmp_seq cache 20; diff --git a/source/core/output_buffers/ut_output_bulk_buffer.tpb b/source/core/output_buffers/ut_output_bulk_buffer.tpb index ceae07862..cfc173e95 100644 --- a/source/core/output_buffers/ut_output_bulk_buffer.tpb +++ b/source/core/output_buffers/ut_output_bulk_buffer.tpb @@ -34,9 +34,8 @@ create or replace type body ut_output_bulk_buffer is a_item_type ); else - self.last_write_message_id := self.last_write_message_id + 1; insert /*+ no_parallel */ into ut_output_buffer_tmp(output_id, message_id, text, item_type) - values (self.output_id, self.last_write_message_id, a_text, a_item_type); + values (self.output_id, ut_output_buffer_tmp_seq.nextval, a_text, a_item_type); end if; commit; end if; @@ -46,10 +45,9 @@ create or replace type body ut_output_bulk_buffer is pragma autonomous_transaction; begin insert /*+ no_parallel */ into ut_output_buffer_tmp(output_id, message_id, text, item_type) - select /*+ no_parallel */ self.output_id, self.last_write_message_id + rownum, t.column_value, a_item_type + select /*+ no_parallel */ self.output_id, ut_output_buffer_tmp_seq.nextval, t.column_value, a_item_type from table(a_text_list) t where t.column_value is not null or a_item_type is not null; - self.last_write_message_id := self.last_write_message_id + sql%rowcount; commit; end; @@ -65,9 +63,8 @@ create or replace type body ut_output_bulk_buffer is a_item_type ); else - self.last_write_message_id := self.last_write_message_id + 1; insert /*+ no_parallel */ into ut_output_buffer_tmp(output_id, message_id, text, item_type) - values (self.output_id, self.last_write_message_id, a_text, a_item_type); + values (self.output_id, ut_output_buffer_tmp_seq.nextval, a_text, a_item_type); end if; commit; end if; diff --git a/source/core/output_buffers/ut_output_clob_buffer_tmp.sql b/source/core/output_buffers/ut_output_clob_buffer_tmp.sql index 40dda121f..9ff9f7413 100644 --- a/source/core/output_buffers/ut_output_clob_buffer_tmp.sql +++ b/source/core/output_buffers/ut_output_clob_buffer_tmp.sql @@ -45,3 +45,5 @@ begin end; end; / + +create sequence ut_output_clob_buffer_tmp_seq cache 20; diff --git a/source/core/output_buffers/ut_output_clob_table_buffer.tpb b/source/core/output_buffers/ut_output_clob_table_buffer.tpb index c4cfdb059..3d49ab08d 100644 --- a/source/core/output_buffers/ut_output_clob_table_buffer.tpb +++ b/source/core/output_buffers/ut_output_clob_table_buffer.tpb @@ -26,9 +26,8 @@ create or replace type body ut_output_clob_table_buffer is pragma autonomous_transaction; begin if a_text is not null or a_item_type is not null then - self.last_write_message_id := self.last_write_message_id + 1; insert /*+ no_parallel */ into ut_output_clob_buffer_tmp(output_id, message_id, text, item_type) - values (self.output_id, self.last_write_message_id, a_text, a_item_type); + values (self.output_id, ut_output_clob_buffer_tmp_seq.nextval, a_text, a_item_type); end if; commit; end; @@ -37,10 +36,9 @@ create or replace type body ut_output_clob_table_buffer is pragma autonomous_transaction; begin insert /*+ no_parallel */ into ut_output_clob_buffer_tmp(output_id, message_id, text, item_type) - select /*+ no_parallel */ self.output_id, self.last_write_message_id + rownum, t.column_value, a_item_type + select /*+ no_parallel */ self.output_id, ut_output_clob_buffer_tmp_seq.nextval, t.column_value, a_item_type from table(a_text_list) t where t.column_value is not null or a_item_type is not null; - self.last_write_message_id := self.last_write_message_id + SQL%rowcount; commit; end; @@ -48,9 +46,8 @@ create or replace type body ut_output_clob_table_buffer is pragma autonomous_transaction; begin if a_text is not null and a_text != empty_clob() or a_item_type is not null then - self.last_write_message_id := self.last_write_message_id + 1; insert /*+ no_parallel */ into ut_output_clob_buffer_tmp(output_id, message_id, text, item_type) - values (self.output_id, self.last_write_message_id, a_text, a_item_type); + values (self.output_id, ut_output_clob_buffer_tmp_seq.nextval, a_text, a_item_type); end if; commit; end; @@ -60,7 +57,6 @@ create or replace type body ut_output_clob_table_buffer is l_buffer_rowids ut_varchar2_rows; l_buffer_data ut_output_data_rows; l_finished_flags ut_integer_list; - l_last_read_message_id integer; l_already_waited_sec number(10,2) := 0; l_finished boolean := false; l_sleep_time number(2,1); @@ -68,25 +64,22 @@ create or replace type body ut_output_clob_table_buffer is l_producer_started boolean := false; l_producer_finished boolean := false; procedure get_data_from_buffer_table( - a_last_read_message_id in out nocopy integer, a_buffer_data out nocopy ut_output_data_rows, a_buffer_rowids out nocopy ut_varchar2_rows, a_finished_flags out nocopy ut_integer_list ) is lc_bulk_limit constant integer := 5000; begin - a_last_read_message_id := coalesce(a_last_read_message_id, 0); with ordered_buffer as ( select /*+ no_parallel index(a) */ ut_output_data_row(a.text, a.item_type), rowidtochar(a.rowid), is_finished from ut_output_clob_buffer_tmp a where a.output_id = self.output_id - and a.message_id <= a_last_read_message_id + lc_bulk_limit + and a.message_id <= (select min(message_id) from ut_output_clob_buffer_tmp o where o.output_id = self.output_id) + lc_bulk_limit order by a.message_id ) select /*+ no_parallel */ b.* bulk collect into a_buffer_data, a_buffer_rowids, a_finished_flags from ordered_buffer b; - a_last_read_message_id := a_last_read_message_id + a_finished_flags.count; end; procedure remove_read_data(a_buffer_rowids ut_varchar2_rows) is @@ -103,7 +96,7 @@ create or replace type body ut_output_clob_table_buffer is l_sleep_time := case when l_already_waited_sec >= 1 then 0.5 else 0.1 end; l_lock_status := self.get_lock_status(); - get_data_from_buffer_table( l_last_read_message_id, l_buffer_data, l_buffer_rowids, l_finished_flags ); + get_data_from_buffer_table( l_buffer_data, l_buffer_rowids, l_finished_flags ); if l_buffer_data.count > 0 then l_already_waited_sec := 0; diff --git a/source/core/output_buffers/ut_output_table_buffer.tpb b/source/core/output_buffers/ut_output_table_buffer.tpb index 480ae9144..e8e2442a7 100644 --- a/source/core/output_buffers/ut_output_table_buffer.tpb +++ b/source/core/output_buffers/ut_output_table_buffer.tpb @@ -26,7 +26,7 @@ create or replace type body ut_output_table_buffer is pragma autonomous_transaction; begin if a_text is not null or a_item_type is not null then - if length(a_text) > ut_utils.gc_max_storage_varchar2_len then + if lengthb(a_text) > ut_utils.gc_max_storage_varchar2_len then self.send_lines( ut_utils.convert_collection( ut_utils.clob_to_table(a_text, ut_utils.gc_max_storage_varchar2_len) @@ -34,9 +34,8 @@ create or replace type body ut_output_table_buffer is a_item_type ); else - self.last_write_message_id := self.last_write_message_id + 1; insert /*+ no_parallel */ into ut_output_buffer_tmp(output_id, message_id, text, item_type) - values (self.output_id, self.last_write_message_id, a_text, a_item_type); + values (self.output_id, ut_output_buffer_tmp_seq.nextval, a_text, a_item_type); end if; commit; end if; @@ -46,10 +45,9 @@ create or replace type body ut_output_table_buffer is pragma autonomous_transaction; begin insert /*+ no_parallel */ into ut_output_buffer_tmp(output_id, message_id, text, item_type) - select /*+ no_parallel */ self.output_id, self.last_write_message_id + rownum, t.column_value, a_item_type + select /*+ no_parallel */ self.output_id, ut_output_buffer_tmp_seq.nextval, t.column_value, a_item_type from table(a_text_list) t where t.column_value is not null or a_item_type is not null; - self.last_write_message_id := self.last_write_message_id + SQL%rowcount; commit; end; @@ -57,7 +55,7 @@ create or replace type body ut_output_table_buffer is pragma autonomous_transaction; begin if a_text is not null and a_text != empty_clob() or a_item_type is not null then - if length(a_text) > ut_utils.gc_max_storage_varchar2_len then + if ut_utils.lengthb_clob(a_text) > ut_utils.gc_max_storage_varchar2_len then self.send_lines( ut_utils.convert_collection( ut_utils.clob_to_table(a_text, ut_utils.gc_max_storage_varchar2_len) @@ -65,9 +63,8 @@ create or replace type body ut_output_table_buffer is a_item_type ); else - self.last_write_message_id := self.last_write_message_id + 1; insert /*+ no_parallel */ into ut_output_buffer_tmp(output_id, message_id, text, item_type) - values (self.output_id, self.last_write_message_id, a_text, a_item_type); + values (self.output_id, ut_output_buffer_tmp_seq.nextval, a_text, a_item_type); end if; commit; end if; @@ -99,7 +96,6 @@ create or replace type body ut_output_table_buffer is l_buffer_texts ut_varchar2_rows; l_buffer_item_types ut_varchar2_rows; l_finished_flags ut_integer_list; - l_last_read_message_id integer; l_already_waited_sec number(10,2) := 0; l_finished boolean := false; l_sleep_time number(2,1); @@ -108,7 +104,6 @@ create or replace type body ut_output_table_buffer is l_producer_finished boolean := false; procedure get_data_from_buffer_table( - a_last_read_message_id in out nocopy integer, a_buffer_texts out nocopy ut_varchar2_rows, a_buffer_item_types out nocopy ut_varchar2_rows, a_finished_flags out nocopy ut_integer_list @@ -116,17 +111,15 @@ create or replace type body ut_output_table_buffer is lc_bulk_limit constant integer := 20000; pragma autonomous_transaction; begin - a_last_read_message_id := coalesce(a_last_read_message_id,0); delete /*+ no_parallel */ from ( select /*+ no_parallel */ * - from ut_output_buffer_tmp o - where o.output_id = self.output_id - and o.message_id <= a_last_read_message_id + lc_bulk_limit - order by o.message_id + from ut_output_buffer_tmp a + where a.output_id = self.output_id + and a.message_id <= (select min(message_id) from ut_output_buffer_tmp o where o.output_id = self.output_id) + lc_bulk_limit + order by a.message_id ) d returning d.text, d.item_type, d.is_finished bulk collect into a_buffer_texts, a_buffer_item_types, a_finished_flags; - a_last_read_message_id := a_last_read_message_id + a_finished_flags.count; commit; end; begin @@ -134,7 +127,7 @@ create or replace type body ut_output_table_buffer is l_sleep_time := case when l_already_waited_sec >= 1 then 0.5 else 0.1 end; l_lock_status := self.get_lock_status(); - get_data_from_buffer_table( l_last_read_message_id, l_buffer_texts, l_buffer_item_types, l_finished_flags ); + get_data_from_buffer_table( l_buffer_texts, l_buffer_item_types, l_finished_flags ); if l_buffer_texts.count > 0 then l_already_waited_sec := 0; diff --git a/source/core/ut_utils.pkb b/source/core/ut_utils.pkb index f29e3b188..81d47221b 100644 --- a/source/core/ut_utils.pkb +++ b/source/core/ut_utils.pkb @@ -92,8 +92,8 @@ create or replace package body ut_utils is a_max_output_len in number := gc_max_output_string_length ) return varchar2 is l_result varchar2(32767); - c_length constant integer := coalesce( length( a_value ), 0 ); - c_max_input_string_length constant integer := a_max_output_len - coalesce( length( a_quote_char ) * 2, 0 ); + c_length constant integer := coalesce( lengthb( a_value ), 0 ); + c_max_input_string_length constant integer := a_max_output_len - coalesce( lengthb( a_quote_char ) * 2, 0 ); c_overflow_substr_len constant integer := c_max_input_string_length - gc_more_data_string_len; begin if c_length = 0 then @@ -112,8 +112,8 @@ create or replace package body ut_utils is a_max_output_len in number := gc_max_output_string_length ) return varchar2 is l_result varchar2(32767); - c_length constant integer := coalesce(dbms_lob.getlength(a_value), 0); - c_max_input_string_length constant integer := a_max_output_len - coalesce( length( a_quote_char ) * 2, 0 ); + c_length constant integer := coalesce(ut_utils.lengthb_clob(a_value), 0); + c_max_input_string_length constant integer := a_max_output_len - coalesce( lengthb( a_quote_char ) * 2, 0 ); c_overflow_substr_len constant integer := c_max_input_string_length - gc_more_data_string_len; begin if a_value is null then @@ -135,7 +135,7 @@ create or replace package body ut_utils is ) return varchar2 is l_result varchar2(32767); c_length constant integer := coalesce(dbms_lob.getlength(a_value), 0); - c_max_input_string_length constant integer := a_max_output_len - coalesce( length( a_quote_char ) * 2, 0 ); + c_max_input_string_length constant integer := a_max_output_len - coalesce( lengthb( a_quote_char ) * 2, 0 ); c_overflow_substr_len constant integer := c_max_input_string_length - gc_more_data_string_len; begin if a_value is null then @@ -412,7 +412,7 @@ create or replace package body ut_utils is if a_list is null then a_list := ut_varchar2_rows(); end if; - if length(a_item) > gc_max_storage_varchar2_len then + if lengthb(a_item) > gc_max_storage_varchar2_len then append_to_list( a_list, ut_utils.convert_collection( @@ -468,7 +468,7 @@ create or replace package body ut_utils is l_result := ut_varchar2_rows(); for i in 1 .. a_collection.count loop l_result.extend(); - l_result(i) := substr(a_collection(i),1,gc_max_storage_varchar2_len); + l_result(i) := substrb(a_collection(i),1,gc_max_storage_varchar2_len); end loop; end if; return l_result; @@ -990,5 +990,39 @@ create or replace package body ut_utils is return l_result; end; + + + /* + * Inspired by + * https://stackoverflow.com/a/48782891 + */ + function lengthb_clob( a_clob clob) return integer is + l_blob blob; + l_desc_offset PLS_INTEGER := 1; + l_src_offset PLS_INTEGER := 1; + l_lang PLS_INTEGER := 0; + l_warning PLS_INTEGER := 0; + l_result integer; + begin + if a_clob = empty_clob() then + l_result := 0; + elsif a_clob is not null then + dbms_lob.createtemporary(l_blob,true); + dbms_lob.converttoblob + ( l_blob + , a_clob + , dbms_lob.getlength(a_clob) + , l_desc_offset + , l_src_offset + , dbms_lob.default_csid + , l_lang + , l_warning + ); + l_result := length(l_blob); + dbms_lob.freetemporary(l_blob); + end if; + return l_result; + end; + end ut_utils; / diff --git a/source/core/ut_utils.pks b/source/core/ut_utils.pks index 90c165bbb..ee5ed7876 100644 --- a/source/core/ut_utils.pks +++ b/source/core/ut_utils.pks @@ -476,6 +476,11 @@ create or replace package ut_utils authid definer is * Return value of interval in plain english */ function interval_to_text(a_interval yminterval_unconstrained) return varchar2; + + /* + * Return length of CLOB in bytes. Null for NULL + */ + function lengthb_clob( a_clob clob) return integer; end ut_utils; / diff --git a/source/uninstall_objects.sql b/source/uninstall_objects.sql index f488b8b8d..a0b53ba21 100644 --- a/source/uninstall_objects.sql +++ b/source/uninstall_objects.sql @@ -333,8 +333,12 @@ drop type ut_output_buffer_base force; drop table ut_output_buffer_tmp purge; +drop table ut_output_buffer_tmp_seq purge; + drop table ut_output_clob_buffer_tmp purge; +drop table ut_output_clob_buffer_tmp_seq purge; + drop table ut_output_buffer_info_tmp purge; drop package ut_session_context; diff --git a/test/ut3_tester/core/test_output_buffer.pkb b/test/ut3_tester/core/test_output_buffer.pkb index edb10e3e6..317e7e3d5 100644 --- a/test/ut3_tester/core/test_output_buffer.pkb +++ b/test/ut3_tester/core/test_output_buffer.pkb @@ -56,8 +56,8 @@ create or replace package body test_output_buffer is procedure test_doesnt_send_on_null_elem is - l_cur sys_refcursor; - l_result integer; + l_actual sys_refcursor; + l_expected sys_refcursor; l_buffer ut3_develop.ut_output_buffer_base := ut3_develop.ut_output_table_buffer(); l_message_id varchar2(255); l_text varchar2(4000); @@ -67,9 +67,12 @@ create or replace package body test_output_buffer is l_buffer.send_lines(ut3_develop.ut_varchar2_rows(null)); l_buffer.send_lines(ut3_develop.ut_varchar2_rows('test')); - select message_id, text into l_message_id, l_text from table(ut3_tester_helper.run_helper.ut_output_buffer_tmp); - ut.expect(l_message_id).to_equal('1'); - ut.expect(l_text).to_equal('test'); + open l_actual for + select text from table(ut3_tester_helper.run_helper.ut_output_buffer_tmp); + open l_expected for + select 'test' as text from dual; + + ut.expect(l_actual).to_equal(l_expected); end; procedure test_send_line is @@ -157,5 +160,41 @@ create or replace package body test_output_buffer is test_purge(ut3_develop.ut_output_clob_table_buffer()); end; -end test_output_buffer; + procedure text_buffer_send_multibyte is + l_input varchar2(32767); + l_max_len integer := ut3_develop.ut_utils.gc_max_storage_varchar2_len; + l_buffer ut3_develop.ut_output_buffer_base := ut3_develop.ut_output_table_buffer(); + l_text varchar2(4000); + begin + --Arrange + ut3_tester_helper.run_helper.delete_buffer(); + l_input := rpad( '❤', l_max_len, 'a' ); + ut.expect( lengthb( l_input ) ).to_be_greater_than(l_max_len); + + --Act + l_buffer.send_line(l_input); + --Assert + select text into l_text from table(ut3_tester_helper.run_helper.ut_output_buffer_tmp); + ut.expect(lengthb(l_text)).to_be_less_or_equal(l_max_len); + end; + + procedure text_buffer_send_clob_multib is + l_input clob; + l_max_len integer := ut3_develop.ut_utils.gc_max_storage_varchar2_len; + l_buffer ut3_develop.ut_output_buffer_base := ut3_develop.ut_output_table_buffer(); + l_text varchar2(4000); + begin + --Arrange + ut3_tester_helper.run_helper.delete_buffer(); + l_input := rpad( '❤', l_max_len, 'a' ); + ut.expect( ut3_develop.ut_utils.lengthb_clob( l_input ) ).to_be_greater_than(l_max_len); + + --Act + l_buffer.send_clob(l_input); + --Assert + select text into l_text from table(ut3_tester_helper.run_helper.ut_output_buffer_tmp); + ut.expect(lengthb(l_text)).to_be_less_or_equal(l_max_len); + end; + + end test_output_buffer; / diff --git a/test/ut3_tester/core/test_output_buffer.pks b/test/ut3_tester/core/test_output_buffer.pks index feaa337f8..5b6cd678b 100644 --- a/test/ut3_tester/core/test_output_buffer.pks +++ b/test/ut3_tester/core/test_output_buffer.pks @@ -47,5 +47,11 @@ create or replace package test_output_buffer is --%test(Purges clob buffer data older than one day and leaves the rest) procedure test_purge_clob_buffer; + --%test(Successfully sends multibyte long line into text buffer) + procedure text_buffer_send_multibyte; + + --%test(Successfully sends multibyte long clob line into text buffer) + procedure text_buffer_send_clob_multib; + end test_output_buffer; / diff --git a/test/ut3_tester/core/test_ut_utils.pkb b/test/ut3_tester/core/test_ut_utils.pkb index ec7e4f403..27c164e4e 100644 --- a/test/ut3_tester/core/test_ut_utils.pkb +++ b/test/ut3_tester/core/test_ut_utils.pkb @@ -489,5 +489,23 @@ end; ut.expect(l_expected).to_equal(l_actual); end; + procedure convert_collection_multibyte is + l_input ut3_develop.ut_varchar2_list; + l_max_len integer := ut3_develop.ut_utils.gc_max_storage_varchar2_len; + begin + --Arrange + l_input := ut3_develop.ut_varchar2_list( rpad( '❤', l_max_len, 'a' ) ); + ut.expect( lengthb( l_input( 1 ) ) ).to_be_greater_than(l_max_len); + + --Act + ut.expect( lengthb( ut3_develop.ut_utils.convert_collection(l_input)(1) ) ).to_be_less_or_equal(l_max_len); + end; + + procedure lengthb_gives_length_in_bytes is + l_clob clob; + begin + l_clob := '❤'; + ut.expect(ut3_develop.ut_utils.lengthb_clob(l_clob)).to_equal(3); + end; end test_ut_utils; / diff --git a/test/ut3_tester/core/test_ut_utils.pks b/test/ut3_tester/core/test_ut_utils.pks index 114b35e86..ca2e7b304 100644 --- a/test/ut3_tester/core/test_ut_utils.pks +++ b/test/ut3_tester/core/test_ut_utils.pks @@ -152,10 +152,15 @@ create or replace package test_ut_utils is procedure int_conv_ym_month; --%test(returns text representation of interval year to month for custom interval) - procedure int_conv_ym_date; - - + procedure int_conv_ym_date; + --%endcontext - + + --%test(convert_collection does not fail on multibyte strings - Issue #1245 ) + procedure convert_collection_multibyte; + + --%test(lengthb returns length of a CLOB in bytes ) + procedure lengthb_gives_length_in_bytes; + end test_ut_utils; / From b70ee1e742652da3c4a13231e1edcc26ecff5998 Mon Sep 17 00:00:00 2001 From: Jacek Gebal Date: Sun, 28 May 2023 17:03:28 +0300 Subject: [PATCH 62/77] Fixed failing unit test and the uninstall script --- source/uninstall_objects.sql | 4 ++-- test/ut3_tester/core/test_ut_utils.pkb | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/source/uninstall_objects.sql b/source/uninstall_objects.sql index a0b53ba21..dde9a824f 100644 --- a/source/uninstall_objects.sql +++ b/source/uninstall_objects.sql @@ -333,11 +333,11 @@ drop type ut_output_buffer_base force; drop table ut_output_buffer_tmp purge; -drop table ut_output_buffer_tmp_seq purge; +drop sequence ut_output_buffer_tmp_seq; drop table ut_output_clob_buffer_tmp purge; -drop table ut_output_clob_buffer_tmp_seq purge; +drop sequence ut_output_clob_buffer_tmp_seq; drop table ut_output_buffer_info_tmp purge; diff --git a/test/ut3_tester/core/test_ut_utils.pkb b/test/ut3_tester/core/test_ut_utils.pkb index 27c164e4e..433987c01 100644 --- a/test/ut3_tester/core/test_ut_utils.pkb +++ b/test/ut3_tester/core/test_ut_utils.pkb @@ -505,7 +505,7 @@ end; l_clob clob; begin l_clob := '❤'; - ut.expect(ut3_develop.ut_utils.lengthb_clob(l_clob)).to_equal(3); + ut.expect(ut3_develop.ut_utils.lengthb_clob(l_clob)).to_be_greater_than(1); end; end test_ut_utils; / From 21462fcfa53b2d466fd7719f9f854facb06c2cdc Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" Date: Sun, 28 May 2023 15:44:45 +0100 Subject: [PATCH 63/77] Updated project version after build [skip ci] --- docs/about/authors.md | 2 +- docs/about/license.md | 2 +- docs/about/project-details.md | 2 +- docs/about/support.md | 2 +- docs/index.md | 2 +- docs/userguide/advanced_data_comparison.md | 2 +- docs/userguide/annotations.md | 2 +- docs/userguide/best-practices.md | 2 +- docs/userguide/coverage.md | 2 +- docs/userguide/exception-reporting.md | 2 +- docs/userguide/expectations.md | 2 +- docs/userguide/getting-started.md | 2 +- docs/userguide/install.md | 2 +- docs/userguide/querying_suites.md | 2 +- docs/userguide/reporters.md | 2 +- docs/userguide/running-unit-tests.md | 2 +- docs/userguide/upgrade.md | 2 +- source/core/ut_utils.pks | 2 +- 18 files changed, 18 insertions(+), 18 deletions(-) diff --git a/docs/about/authors.md b/docs/about/authors.md index 9d18766ae..d4bebf085 100644 --- a/docs/about/authors.md +++ b/docs/about/authors.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4171--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4176--develop-blue.svg) ### utPLSQL v3 Major Contributors diff --git a/docs/about/license.md b/docs/about/license.md index a9cd17bc9..adf03ba11 100644 --- a/docs/about/license.md +++ b/docs/about/license.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4171--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4176--develop-blue.svg) # Version Information diff --git a/docs/about/project-details.md b/docs/about/project-details.md index 0b1b005c6..aca73a1d6 100644 --- a/docs/about/project-details.md +++ b/docs/about/project-details.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4171--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4176--develop-blue.svg) # utPLSQL Project Details diff --git a/docs/about/support.md b/docs/about/support.md index 639dc2970..c83158ec0 100644 --- a/docs/about/support.md +++ b/docs/about/support.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4171--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4176--develop-blue.svg) # How to get support diff --git a/docs/index.md b/docs/index.md index bd908d60c..f60d80aa9 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4171--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4176--develop-blue.svg) ## What is utPLSQL diff --git a/docs/userguide/advanced_data_comparison.md b/docs/userguide/advanced_data_comparison.md index eb3b311a5..5fc69e002 100644 --- a/docs/userguide/advanced_data_comparison.md +++ b/docs/userguide/advanced_data_comparison.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4171--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4176--develop-blue.svg) # Advanced data comparison diff --git a/docs/userguide/annotations.md b/docs/userguide/annotations.md index 4f1c6b420..4b3af7c2a 100644 --- a/docs/userguide/annotations.md +++ b/docs/userguide/annotations.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4171--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4176--develop-blue.svg) Annotations are used to configure tests and suites in a declarative way similar to modern OOP languages. This way, test configuration is stored along with the test logic inside the test package. No additional configuration files or tables are needed for test cases. The annotation names are based on popular testing frameworks such as JUnit. diff --git a/docs/userguide/best-practices.md b/docs/userguide/best-practices.md index f5f889ca6..6c1920c6b 100644 --- a/docs/userguide/best-practices.md +++ b/docs/userguide/best-practices.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4171--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4176--develop-blue.svg) The following are best practices we at utPLSQL have learned about PL/SQL and Unit Testing. diff --git a/docs/userguide/coverage.md b/docs/userguide/coverage.md index dbd584a7c..0466e237a 100644 --- a/docs/userguide/coverage.md +++ b/docs/userguide/coverage.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4171--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4176--develop-blue.svg) utPLSQL comes with a built-in coverage reporting engine. The code coverage reporting uses package DBMS_PROFILER (and DBMS_PLSQL_CODE_COVERAGE on Oracle database version 12.2 and above) provided with Oracle database. Code coverage is gathered for the following source types: diff --git a/docs/userguide/exception-reporting.md b/docs/userguide/exception-reporting.md index cad56f1b8..5b242a0cd 100644 --- a/docs/userguide/exception-reporting.md +++ b/docs/userguide/exception-reporting.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4171--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4176--develop-blue.svg) utPLSQL is responsible for handling exceptions wherever they occur in the test run. The framework is trapping most of the exceptions so that the test execution is not affected by individual tests or test packages throwing an exception. The framework provides a full stacktrace for every exception that was thrown. The reported stacktrace does not include any utPLSQL library calls in it. diff --git a/docs/userguide/expectations.md b/docs/userguide/expectations.md index 67f851f36..d188c0aa2 100644 --- a/docs/userguide/expectations.md +++ b/docs/userguide/expectations.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4171--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4176--develop-blue.svg) ## Expectation concepts diff --git a/docs/userguide/getting-started.md b/docs/userguide/getting-started.md index b2c67054e..1a0aceeb9 100644 --- a/docs/userguide/getting-started.md +++ b/docs/userguide/getting-started.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4171--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4176--develop-blue.svg) # Getting started with TDD and utPLSQL diff --git a/docs/userguide/install.md b/docs/userguide/install.md index 4e1b3e903..e12ed3588 100644 --- a/docs/userguide/install.md +++ b/docs/userguide/install.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4171--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4176--develop-blue.svg) ## Supported database versions diff --git a/docs/userguide/querying_suites.md b/docs/userguide/querying_suites.md index d68240a5d..d7f2a2838 100644 --- a/docs/userguide/querying_suites.md +++ b/docs/userguide/querying_suites.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4171--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4176--develop-blue.svg) ## Obtaining information about suites diff --git a/docs/userguide/reporters.md b/docs/userguide/reporters.md index ca3878f9f..a804d98e7 100644 --- a/docs/userguide/reporters.md +++ b/docs/userguide/reporters.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4171--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4176--develop-blue.svg) utPLSQL provides several reporting formats. The sections below describe most of them. diff --git a/docs/userguide/running-unit-tests.md b/docs/userguide/running-unit-tests.md index 4fdb74073..282e07d09 100644 --- a/docs/userguide/running-unit-tests.md +++ b/docs/userguide/running-unit-tests.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4171--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4176--develop-blue.svg) utPLSQL framework provides two main entry points to run unit tests from within the database: diff --git a/docs/userguide/upgrade.md b/docs/userguide/upgrade.md index bd401f5d9..bbca978ba 100644 --- a/docs/userguide/upgrade.md +++ b/docs/userguide/upgrade.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4171--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4176--develop-blue.svg) # Upgrading from version 2 diff --git a/source/core/ut_utils.pks b/source/core/ut_utils.pks index ee5ed7876..552f31143 100644 --- a/source/core/ut_utils.pks +++ b/source/core/ut_utils.pks @@ -21,7 +21,7 @@ create or replace package ut_utils authid definer is * */ - gc_version constant varchar2(50) := 'v3.1.14.4171-develop'; + gc_version constant varchar2(50) := 'v3.1.14.4176-develop'; subtype t_executable_type is varchar2(30); gc_before_all constant t_executable_type := 'beforeall'; From cd3e2b940208c6f9f68d6ae74b10d86019761c3f Mon Sep 17 00:00:00 2001 From: Jonas Gassenmeyer Date: Fri, 7 Jul 2023 14:17:00 +0200 Subject: [PATCH 64/77] fix(doc): Update running-unit-tests.md - remove example --- docs/userguide/running-unit-tests.md | 10 +--------- 1 file changed, 1 insertion(+), 9 deletions(-) diff --git a/docs/userguide/running-unit-tests.md b/docs/userguide/running-unit-tests.md index 282e07d09..447b5eb54 100644 --- a/docs/userguide/running-unit-tests.md +++ b/docs/userguide/running-unit-tests.md @@ -120,14 +120,6 @@ end; ``` Executes all tests from package _hr.test_apply_bonus_ and all tests from schema _cust_. -```sql linenums="1" -set serveroutput on -begin - ut.run(ut_varchar2_list('hr.test_apply_bonus,cust)'); -end; -``` - -Executes all tests from package _hr.test_apply_bonus_ and all tests from schema _cust_. ```sql linenums="1" set serveroutput on @@ -566,4 +558,4 @@ Example call with characterset provided: begin ut.run('hr.test_apply_bonus', ut_junit_reporter(), a_client_character_set => 'Windows-1251'); end; -``` \ No newline at end of file +``` From e4d8528822254212a28e676a11ba9d578cfeee04 Mon Sep 17 00:00:00 2001 From: Jacek Gebal Date: Sun, 9 Jul 2023 19:11:21 +0300 Subject: [PATCH 65/77] Added missing grant for `ut_output_bulk_buffer`. --- source/create_grants.sql | 1 + 1 file changed, 1 insertion(+) diff --git a/source/create_grants.sql b/source/create_grants.sql index 8a44ab371..0e9f46cc5 100644 --- a/source/create_grants.sql +++ b/source/create_grants.sql @@ -123,6 +123,7 @@ grant execute on &&ut3_owner..ut_console_reporter_base to &ut3_user; grant execute on &&ut3_owner..ut_output_buffer_base to &ut3_user; grant execute on &&ut3_owner..ut_output_table_buffer to &ut3_user; grant execute on &&ut3_owner..ut_output_clob_table_buffer to &ut3_user; +grant execute on &&ut3_owner..ut_output_bulk_buffer to &ut3_user; --needed internally for selecting from annotation objects within packages that use invoker rights grant execute on &&ut3_owner..ut_annotation_objs_cache_info to &ut3_user; From 550ea328e74cc211408292b1f481b4feabe5ce46 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" Date: Sun, 9 Jul 2023 17:20:41 +0100 Subject: [PATCH 66/77] Updated project version after build [skip ci] --- docs/about/authors.md | 2 +- docs/about/license.md | 2 +- docs/about/project-details.md | 2 +- docs/about/support.md | 2 +- docs/index.md | 2 +- docs/userguide/advanced_data_comparison.md | 2 +- docs/userguide/annotations.md | 2 +- docs/userguide/best-practices.md | 2 +- docs/userguide/coverage.md | 2 +- docs/userguide/exception-reporting.md | 2 +- docs/userguide/expectations.md | 2 +- docs/userguide/getting-started.md | 2 +- docs/userguide/install.md | 2 +- docs/userguide/querying_suites.md | 2 +- docs/userguide/reporters.md | 2 +- docs/userguide/running-unit-tests.md | 2 +- docs/userguide/upgrade.md | 2 +- source/core/ut_utils.pks | 2 +- 18 files changed, 18 insertions(+), 18 deletions(-) diff --git a/docs/about/authors.md b/docs/about/authors.md index d4bebf085..cd5f1191e 100644 --- a/docs/about/authors.md +++ b/docs/about/authors.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4176--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4178--develop-blue.svg) ### utPLSQL v3 Major Contributors diff --git a/docs/about/license.md b/docs/about/license.md index adf03ba11..f81d13395 100644 --- a/docs/about/license.md +++ b/docs/about/license.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4176--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4178--develop-blue.svg) # Version Information diff --git a/docs/about/project-details.md b/docs/about/project-details.md index aca73a1d6..e315c921f 100644 --- a/docs/about/project-details.md +++ b/docs/about/project-details.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4176--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4178--develop-blue.svg) # utPLSQL Project Details diff --git a/docs/about/support.md b/docs/about/support.md index c83158ec0..968d9acc4 100644 --- a/docs/about/support.md +++ b/docs/about/support.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4176--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4178--develop-blue.svg) # How to get support diff --git a/docs/index.md b/docs/index.md index f60d80aa9..49f5d42a7 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4176--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4178--develop-blue.svg) ## What is utPLSQL diff --git a/docs/userguide/advanced_data_comparison.md b/docs/userguide/advanced_data_comparison.md index 5fc69e002..325b831c5 100644 --- a/docs/userguide/advanced_data_comparison.md +++ b/docs/userguide/advanced_data_comparison.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4176--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4178--develop-blue.svg) # Advanced data comparison diff --git a/docs/userguide/annotations.md b/docs/userguide/annotations.md index 4b3af7c2a..95b34accc 100644 --- a/docs/userguide/annotations.md +++ b/docs/userguide/annotations.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4176--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4178--develop-blue.svg) Annotations are used to configure tests and suites in a declarative way similar to modern OOP languages. This way, test configuration is stored along with the test logic inside the test package. No additional configuration files or tables are needed for test cases. The annotation names are based on popular testing frameworks such as JUnit. diff --git a/docs/userguide/best-practices.md b/docs/userguide/best-practices.md index 6c1920c6b..94ab25215 100644 --- a/docs/userguide/best-practices.md +++ b/docs/userguide/best-practices.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4176--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4178--develop-blue.svg) The following are best practices we at utPLSQL have learned about PL/SQL and Unit Testing. diff --git a/docs/userguide/coverage.md b/docs/userguide/coverage.md index 0466e237a..e30c87d00 100644 --- a/docs/userguide/coverage.md +++ b/docs/userguide/coverage.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4176--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4178--develop-blue.svg) utPLSQL comes with a built-in coverage reporting engine. The code coverage reporting uses package DBMS_PROFILER (and DBMS_PLSQL_CODE_COVERAGE on Oracle database version 12.2 and above) provided with Oracle database. Code coverage is gathered for the following source types: diff --git a/docs/userguide/exception-reporting.md b/docs/userguide/exception-reporting.md index 5b242a0cd..7fdac2189 100644 --- a/docs/userguide/exception-reporting.md +++ b/docs/userguide/exception-reporting.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4176--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4178--develop-blue.svg) utPLSQL is responsible for handling exceptions wherever they occur in the test run. The framework is trapping most of the exceptions so that the test execution is not affected by individual tests or test packages throwing an exception. The framework provides a full stacktrace for every exception that was thrown. The reported stacktrace does not include any utPLSQL library calls in it. diff --git a/docs/userguide/expectations.md b/docs/userguide/expectations.md index d188c0aa2..ceca41af8 100644 --- a/docs/userguide/expectations.md +++ b/docs/userguide/expectations.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4176--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4178--develop-blue.svg) ## Expectation concepts diff --git a/docs/userguide/getting-started.md b/docs/userguide/getting-started.md index 1a0aceeb9..e5b636f53 100644 --- a/docs/userguide/getting-started.md +++ b/docs/userguide/getting-started.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4176--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4178--develop-blue.svg) # Getting started with TDD and utPLSQL diff --git a/docs/userguide/install.md b/docs/userguide/install.md index e12ed3588..ce21e475f 100644 --- a/docs/userguide/install.md +++ b/docs/userguide/install.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4176--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4178--develop-blue.svg) ## Supported database versions diff --git a/docs/userguide/querying_suites.md b/docs/userguide/querying_suites.md index d7f2a2838..7c4891aea 100644 --- a/docs/userguide/querying_suites.md +++ b/docs/userguide/querying_suites.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4176--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4178--develop-blue.svg) ## Obtaining information about suites diff --git a/docs/userguide/reporters.md b/docs/userguide/reporters.md index a804d98e7..f46d7dba4 100644 --- a/docs/userguide/reporters.md +++ b/docs/userguide/reporters.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4176--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4178--develop-blue.svg) utPLSQL provides several reporting formats. The sections below describe most of them. diff --git a/docs/userguide/running-unit-tests.md b/docs/userguide/running-unit-tests.md index 282e07d09..618e61f1c 100644 --- a/docs/userguide/running-unit-tests.md +++ b/docs/userguide/running-unit-tests.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4176--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4178--develop-blue.svg) utPLSQL framework provides two main entry points to run unit tests from within the database: diff --git a/docs/userguide/upgrade.md b/docs/userguide/upgrade.md index bbca978ba..2eecf9c85 100644 --- a/docs/userguide/upgrade.md +++ b/docs/userguide/upgrade.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4176--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4178--develop-blue.svg) # Upgrading from version 2 diff --git a/source/core/ut_utils.pks b/source/core/ut_utils.pks index 552f31143..7c49dad67 100644 --- a/source/core/ut_utils.pks +++ b/source/core/ut_utils.pks @@ -21,7 +21,7 @@ create or replace package ut_utils authid definer is * */ - gc_version constant varchar2(50) := 'v3.1.14.4176-develop'; + gc_version constant varchar2(50) := 'v3.1.14.4178-develop'; subtype t_executable_type is varchar2(30); gc_before_all constant t_executable_type := 'beforeall'; From fc8c520bfb569cee737193c7db81c7efec07a20b Mon Sep 17 00:00:00 2001 From: Jonas Gassenmeyer Date: Mon, 10 Jul 2023 08:12:46 +0200 Subject: [PATCH 67/77] fix(doc): Update running-unit-tests.md add comments --- docs/userguide/running-unit-tests.md | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/docs/userguide/running-unit-tests.md b/docs/userguide/running-unit-tests.md index e35001a52..fde0330c9 100644 --- a/docs/userguide/running-unit-tests.md +++ b/docs/userguide/running-unit-tests.md @@ -118,7 +118,16 @@ begin ut.run(ut_varchar2_list('hr.test_apply_bonus','cust')); end; ``` -Executes all tests from package _hr.test_apply_bonus_ and all tests from schema _cust_. +Executes all tests from package _hr.test_apply_bonus_ and all tests from schema _cust_ (passing individual items to be executed as elements of the ut_varchar2_list table type). + + +```sql linenums="1" +set serveroutput on +begin + ut.run(ut_varchar2_list('hr.test_apply_bonus,cust')); +end; +``` +Executes all tests from package _hr.test_apply_bonus_ and all tests from schema _cust_ (passing all items as a comma-separated-list of values into a single element of the ut_varchar2_list table type). ```sql linenums="1" @@ -128,7 +137,7 @@ begin end; ``` -Executes all tests from package _hr.test_apply_bonus_ and all tests from schema _cust_. +Executes all tests from package _hr.test_apply_bonus_ and all tests from schema _cust_ (no explicit ut_varchar2_list table type). Using a list of items to execute allows you to execute a fine-grained set of tests. From 242695255a9751d642563bfe75ad02582032561b Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" Date: Tue, 29 Aug 2023 21:51:17 +0100 Subject: [PATCH 68/77] Updated project version after build [skip ci] --- docs/about/authors.md | 2 +- docs/about/license.md | 2 +- docs/about/project-details.md | 2 +- docs/about/support.md | 2 +- docs/index.md | 2 +- docs/userguide/advanced_data_comparison.md | 2 +- docs/userguide/annotations.md | 2 +- docs/userguide/best-practices.md | 2 +- docs/userguide/coverage.md | 2 +- docs/userguide/exception-reporting.md | 2 +- docs/userguide/expectations.md | 2 +- docs/userguide/getting-started.md | 2 +- docs/userguide/install.md | 2 +- docs/userguide/querying_suites.md | 2 +- docs/userguide/reporters.md | 2 +- docs/userguide/running-unit-tests.md | 2 +- docs/userguide/upgrade.md | 2 +- source/core/ut_utils.pks | 2 +- 18 files changed, 18 insertions(+), 18 deletions(-) diff --git a/docs/about/authors.md b/docs/about/authors.md index cd5f1191e..43b94f7ab 100644 --- a/docs/about/authors.md +++ b/docs/about/authors.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4178--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4182--develop-blue.svg) ### utPLSQL v3 Major Contributors diff --git a/docs/about/license.md b/docs/about/license.md index f81d13395..cf10e3bd6 100644 --- a/docs/about/license.md +++ b/docs/about/license.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4178--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4182--develop-blue.svg) # Version Information diff --git a/docs/about/project-details.md b/docs/about/project-details.md index e315c921f..6889d9edb 100644 --- a/docs/about/project-details.md +++ b/docs/about/project-details.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4178--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4182--develop-blue.svg) # utPLSQL Project Details diff --git a/docs/about/support.md b/docs/about/support.md index 968d9acc4..c2af14837 100644 --- a/docs/about/support.md +++ b/docs/about/support.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4178--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4182--develop-blue.svg) # How to get support diff --git a/docs/index.md b/docs/index.md index 49f5d42a7..f22ea814e 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4178--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4182--develop-blue.svg) ## What is utPLSQL diff --git a/docs/userguide/advanced_data_comparison.md b/docs/userguide/advanced_data_comparison.md index 325b831c5..f676ae4db 100644 --- a/docs/userguide/advanced_data_comparison.md +++ b/docs/userguide/advanced_data_comparison.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4178--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4182--develop-blue.svg) # Advanced data comparison diff --git a/docs/userguide/annotations.md b/docs/userguide/annotations.md index 95b34accc..a21f66424 100644 --- a/docs/userguide/annotations.md +++ b/docs/userguide/annotations.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4178--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4182--develop-blue.svg) Annotations are used to configure tests and suites in a declarative way similar to modern OOP languages. This way, test configuration is stored along with the test logic inside the test package. No additional configuration files or tables are needed for test cases. The annotation names are based on popular testing frameworks such as JUnit. diff --git a/docs/userguide/best-practices.md b/docs/userguide/best-practices.md index 94ab25215..f563fb819 100644 --- a/docs/userguide/best-practices.md +++ b/docs/userguide/best-practices.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4178--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4182--develop-blue.svg) The following are best practices we at utPLSQL have learned about PL/SQL and Unit Testing. diff --git a/docs/userguide/coverage.md b/docs/userguide/coverage.md index e30c87d00..4c1e1dcb8 100644 --- a/docs/userguide/coverage.md +++ b/docs/userguide/coverage.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4178--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4182--develop-blue.svg) utPLSQL comes with a built-in coverage reporting engine. The code coverage reporting uses package DBMS_PROFILER (and DBMS_PLSQL_CODE_COVERAGE on Oracle database version 12.2 and above) provided with Oracle database. Code coverage is gathered for the following source types: diff --git a/docs/userguide/exception-reporting.md b/docs/userguide/exception-reporting.md index 7fdac2189..467651334 100644 --- a/docs/userguide/exception-reporting.md +++ b/docs/userguide/exception-reporting.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4178--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4182--develop-blue.svg) utPLSQL is responsible for handling exceptions wherever they occur in the test run. The framework is trapping most of the exceptions so that the test execution is not affected by individual tests or test packages throwing an exception. The framework provides a full stacktrace for every exception that was thrown. The reported stacktrace does not include any utPLSQL library calls in it. diff --git a/docs/userguide/expectations.md b/docs/userguide/expectations.md index ceca41af8..ce9e519d5 100644 --- a/docs/userguide/expectations.md +++ b/docs/userguide/expectations.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4178--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4182--develop-blue.svg) ## Expectation concepts diff --git a/docs/userguide/getting-started.md b/docs/userguide/getting-started.md index e5b636f53..47f2cb92e 100644 --- a/docs/userguide/getting-started.md +++ b/docs/userguide/getting-started.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4178--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4182--develop-blue.svg) # Getting started with TDD and utPLSQL diff --git a/docs/userguide/install.md b/docs/userguide/install.md index ce21e475f..69183c577 100644 --- a/docs/userguide/install.md +++ b/docs/userguide/install.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4178--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4182--develop-blue.svg) ## Supported database versions diff --git a/docs/userguide/querying_suites.md b/docs/userguide/querying_suites.md index 7c4891aea..3bf028596 100644 --- a/docs/userguide/querying_suites.md +++ b/docs/userguide/querying_suites.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4178--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4182--develop-blue.svg) ## Obtaining information about suites diff --git a/docs/userguide/reporters.md b/docs/userguide/reporters.md index f46d7dba4..3c36676e8 100644 --- a/docs/userguide/reporters.md +++ b/docs/userguide/reporters.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4178--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4182--develop-blue.svg) utPLSQL provides several reporting formats. The sections below describe most of them. diff --git a/docs/userguide/running-unit-tests.md b/docs/userguide/running-unit-tests.md index fde0330c9..662b5b004 100644 --- a/docs/userguide/running-unit-tests.md +++ b/docs/userguide/running-unit-tests.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4178--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4182--develop-blue.svg) utPLSQL framework provides two main entry points to run unit tests from within the database: diff --git a/docs/userguide/upgrade.md b/docs/userguide/upgrade.md index 2eecf9c85..ae4499f98 100644 --- a/docs/userguide/upgrade.md +++ b/docs/userguide/upgrade.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4178--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4182--develop-blue.svg) # Upgrading from version 2 diff --git a/source/core/ut_utils.pks b/source/core/ut_utils.pks index 7c49dad67..29ce06991 100644 --- a/source/core/ut_utils.pks +++ b/source/core/ut_utils.pks @@ -21,7 +21,7 @@ create or replace package ut_utils authid definer is * */ - gc_version constant varchar2(50) := 'v3.1.14.4178-develop'; + gc_version constant varchar2(50) := 'v3.1.14.4182-develop'; subtype t_executable_type is varchar2(30); gc_before_all constant t_executable_type := 'beforeall'; From 03106637532a424c413cd34e7614ef10b0359ad0 Mon Sep 17 00:00:00 2001 From: Lukasz Wasylow Date: Fri, 22 Sep 2023 21:07:16 +0100 Subject: [PATCH 69/77] Issue #1268 The line-rate is not recorded for packages and classes. --- .../ut_coverage_cobertura_reporter.tpb | 37 +++++++++++++++---- .../test_cov_cobertura_reporter.pkb | 4 +- .../test_coverage_standalone.pkb | 4 +- 3 files changed, 33 insertions(+), 12 deletions(-) diff --git a/source/reporters/ut_coverage_cobertura_reporter.tpb b/source/reporters/ut_coverage_cobertura_reporter.tpb index 50eb34631..d833ac0aa 100644 --- a/source/reporters/ut_coverage_cobertura_reporter.tpb +++ b/source/reporters/ut_coverage_cobertura_reporter.tpb @@ -29,11 +29,19 @@ create or replace type body ut_coverage_cobertura_reporter is l_report_lines ut_varchar2_list; l_coverage_data ut_coverage.t_coverage; - function get_lines_xml(a_unit_coverage ut_coverage.t_unit_coverage) return clob is + function get_line_rate(a_lines_covered in integer, a_lines_hit in integer) return varchar2 is + begin + return to_char(round((case a_lines_covered when 0 then 0 else a_lines_covered/a_lines_hit end), 17), rpad('FM0.0',20,'9') , 'NLS_NUMERIC_CHARACTERS=''. '''); + end; + + procedure get_lines_xml(a_unit_coverage ut_coverage.t_unit_coverage, + a_lines_result in out nocopy clob, a_lines_hits out number, a_lines_total out number) is l_file_part varchar2(32767); l_result clob; l_line_no binary_integer; l_pct integer; + l_lines_hits integer := 0; + l_lines_total integer := 0; begin dbms_lob.createtemporary(l_result, true); l_line_no := a_unit_coverage.lines.first; @@ -41,11 +49,14 @@ create or replace type body ut_coverage_cobertura_reporter is for i in 1 .. a_unit_coverage.total_lines loop ut_utils.append_to_clob(l_result, ''||chr(10)); end loop; + l_lines_hits:=0; + l_lines_total:= a_unit_coverage.total_lines; else while l_line_no is not null loop if a_unit_coverage.lines(l_line_no).executions = 0 then l_file_part := ''||chr(10); else + l_lines_hits:= l_lines_hits+1; l_file_part := '' + '' ); ut_utils.append_to_list( @@ -129,13 +150,13 @@ create or replace type body ut_coverage_cobertura_reporter is ut_utils.append_to_list( l_result, '' + ||dbms_xmlgen.convert(l_unit)||'" line-rate="'||get_line_rate(l_line_hits,l_line_total)||'" branch-rate="0.0" complexity="0.0">' ); ut_utils.append_to_list(l_result, ''); - - ut_utils.append_to_list( l_result, get_lines_xml(a_coverage_data.objects(l_unit)) ); - + ut_utils.append_to_list( l_result,l_lines_xml); + dbms_lob.freetemporary(l_lines_xml); + ut_utils.append_to_list(l_result, c_lines_footer); ut_utils.append_to_list(l_result, c_class_footer); ut_utils.append_to_list(l_result, c_classes_footer); diff --git a/test/ut3_user/reporters/test_coverage/test_cov_cobertura_reporter.pkb b/test/ut3_user/reporters/test_coverage/test_cov_cobertura_reporter.pkb index 2c36f37ba..ad6c9dbb4 100644 --- a/test/ut3_user/reporters/test_coverage/test_cov_cobertura_reporter.pkb +++ b/test/ut3_user/reporters/test_coverage/test_cov_cobertura_reporter.pkb @@ -25,9 +25,9 @@ create or replace package body test_cov_cobertura_reporter is ]'||l_file_path||q'[ - + - + ]'||l_block_cov||q'[ diff --git a/test/ut3_user/reporters/test_coverage/test_coverage_standalone.pkb b/test/ut3_user/reporters/test_coverage/test_coverage_standalone.pkb index e5706f885..90cfb484e 100644 --- a/test/ut3_user/reporters/test_coverage/test_coverage_standalone.pkb +++ b/test/ut3_user/reporters/test_coverage/test_coverage_standalone.pkb @@ -20,9 +20,9 @@ create or replace package body test_coverage_standalone is ]'||l_file_path||q'[ - + - + ]'||l_block_cov||q'[ From e0a9fa16a973d3a47784f58485238309fb44bb69 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" Date: Sat, 23 Sep 2023 19:28:21 +0100 Subject: [PATCH 70/77] Updated project version after build [skip ci] --- docs/about/authors.md | 2 +- docs/about/license.md | 2 +- docs/about/project-details.md | 2 +- docs/about/support.md | 2 +- docs/index.md | 2 +- docs/userguide/advanced_data_comparison.md | 2 +- docs/userguide/annotations.md | 2 +- docs/userguide/best-practices.md | 2 +- docs/userguide/coverage.md | 2 +- docs/userguide/exception-reporting.md | 2 +- docs/userguide/expectations.md | 2 +- docs/userguide/getting-started.md | 2 +- docs/userguide/install.md | 2 +- docs/userguide/querying_suites.md | 2 +- docs/userguide/reporters.md | 2 +- docs/userguide/running-unit-tests.md | 2 +- docs/userguide/upgrade.md | 2 +- source/core/ut_utils.pks | 2 +- 18 files changed, 18 insertions(+), 18 deletions(-) diff --git a/docs/about/authors.md b/docs/about/authors.md index 43b94f7ab..c4c40bfad 100644 --- a/docs/about/authors.md +++ b/docs/about/authors.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4182--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4187--develop-blue.svg) ### utPLSQL v3 Major Contributors diff --git a/docs/about/license.md b/docs/about/license.md index cf10e3bd6..702ea3f9c 100644 --- a/docs/about/license.md +++ b/docs/about/license.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4182--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4187--develop-blue.svg) # Version Information diff --git a/docs/about/project-details.md b/docs/about/project-details.md index 6889d9edb..60aa6a657 100644 --- a/docs/about/project-details.md +++ b/docs/about/project-details.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4182--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4187--develop-blue.svg) # utPLSQL Project Details diff --git a/docs/about/support.md b/docs/about/support.md index c2af14837..fc412205d 100644 --- a/docs/about/support.md +++ b/docs/about/support.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4182--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4187--develop-blue.svg) # How to get support diff --git a/docs/index.md b/docs/index.md index f22ea814e..8325ec636 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4182--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4187--develop-blue.svg) ## What is utPLSQL diff --git a/docs/userguide/advanced_data_comparison.md b/docs/userguide/advanced_data_comparison.md index f676ae4db..1f7f2d191 100644 --- a/docs/userguide/advanced_data_comparison.md +++ b/docs/userguide/advanced_data_comparison.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4182--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4187--develop-blue.svg) # Advanced data comparison diff --git a/docs/userguide/annotations.md b/docs/userguide/annotations.md index a21f66424..d1ff5da07 100644 --- a/docs/userguide/annotations.md +++ b/docs/userguide/annotations.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4182--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4187--develop-blue.svg) Annotations are used to configure tests and suites in a declarative way similar to modern OOP languages. This way, test configuration is stored along with the test logic inside the test package. No additional configuration files or tables are needed for test cases. The annotation names are based on popular testing frameworks such as JUnit. diff --git a/docs/userguide/best-practices.md b/docs/userguide/best-practices.md index f563fb819..5148273e6 100644 --- a/docs/userguide/best-practices.md +++ b/docs/userguide/best-practices.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4182--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4187--develop-blue.svg) The following are best practices we at utPLSQL have learned about PL/SQL and Unit Testing. diff --git a/docs/userguide/coverage.md b/docs/userguide/coverage.md index 4c1e1dcb8..5d071335e 100644 --- a/docs/userguide/coverage.md +++ b/docs/userguide/coverage.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4182--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4187--develop-blue.svg) utPLSQL comes with a built-in coverage reporting engine. The code coverage reporting uses package DBMS_PROFILER (and DBMS_PLSQL_CODE_COVERAGE on Oracle database version 12.2 and above) provided with Oracle database. Code coverage is gathered for the following source types: diff --git a/docs/userguide/exception-reporting.md b/docs/userguide/exception-reporting.md index 467651334..ca52ac951 100644 --- a/docs/userguide/exception-reporting.md +++ b/docs/userguide/exception-reporting.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4182--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4187--develop-blue.svg) utPLSQL is responsible for handling exceptions wherever they occur in the test run. The framework is trapping most of the exceptions so that the test execution is not affected by individual tests or test packages throwing an exception. The framework provides a full stacktrace for every exception that was thrown. The reported stacktrace does not include any utPLSQL library calls in it. diff --git a/docs/userguide/expectations.md b/docs/userguide/expectations.md index ce9e519d5..b6f3d7e29 100644 --- a/docs/userguide/expectations.md +++ b/docs/userguide/expectations.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4182--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4187--develop-blue.svg) ## Expectation concepts diff --git a/docs/userguide/getting-started.md b/docs/userguide/getting-started.md index 47f2cb92e..9620878ad 100644 --- a/docs/userguide/getting-started.md +++ b/docs/userguide/getting-started.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4182--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4187--develop-blue.svg) # Getting started with TDD and utPLSQL diff --git a/docs/userguide/install.md b/docs/userguide/install.md index 69183c577..8f32c5a10 100644 --- a/docs/userguide/install.md +++ b/docs/userguide/install.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4182--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4187--develop-blue.svg) ## Supported database versions diff --git a/docs/userguide/querying_suites.md b/docs/userguide/querying_suites.md index 3bf028596..e3ec1f53d 100644 --- a/docs/userguide/querying_suites.md +++ b/docs/userguide/querying_suites.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4182--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4187--develop-blue.svg) ## Obtaining information about suites diff --git a/docs/userguide/reporters.md b/docs/userguide/reporters.md index 3c36676e8..e37ac6627 100644 --- a/docs/userguide/reporters.md +++ b/docs/userguide/reporters.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4182--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4187--develop-blue.svg) utPLSQL provides several reporting formats. The sections below describe most of them. diff --git a/docs/userguide/running-unit-tests.md b/docs/userguide/running-unit-tests.md index 662b5b004..3b23f5f98 100644 --- a/docs/userguide/running-unit-tests.md +++ b/docs/userguide/running-unit-tests.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4182--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4187--develop-blue.svg) utPLSQL framework provides two main entry points to run unit tests from within the database: diff --git a/docs/userguide/upgrade.md b/docs/userguide/upgrade.md index ae4499f98..e5537f520 100644 --- a/docs/userguide/upgrade.md +++ b/docs/userguide/upgrade.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4182--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4187--develop-blue.svg) # Upgrading from version 2 diff --git a/source/core/ut_utils.pks b/source/core/ut_utils.pks index 29ce06991..522da1480 100644 --- a/source/core/ut_utils.pks +++ b/source/core/ut_utils.pks @@ -21,7 +21,7 @@ create or replace package ut_utils authid definer is * */ - gc_version constant varchar2(50) := 'v3.1.14.4182-develop'; + gc_version constant varchar2(50) := 'v3.1.14.4187-develop'; subtype t_executable_type is varchar2(30); gc_before_all constant t_executable_type := 'beforeall'; From 0474d7a8f2944b864e8b221e9a7c79f11d23c245 Mon Sep 17 00:00:00 2001 From: Lukasz Wasylow Date: Sun, 24 Sep 2023 23:52:15 +0100 Subject: [PATCH 71/77] First checking. Still some more tests to go for complex nesting. --- source/reporters/ut_tfs_junit_reporter.tpb | 92 ++++++---- .../reporters/test_tfs_junit_reporter.pkb | 161 +++++++++++++++++- .../reporters/test_tfs_junit_reporter.pks | 9 + 3 files changed, 218 insertions(+), 44 deletions(-) diff --git a/source/reporters/ut_tfs_junit_reporter.tpb b/source/reporters/ut_tfs_junit_reporter.tpb index e2d45a369..5b8702808 100644 --- a/source/reporters/ut_tfs_junit_reporter.tpb +++ b/source/reporters/ut_tfs_junit_reporter.tpb @@ -51,11 +51,12 @@ create or replace type body ut_tfs_junit_reporter is return regexp_substr(a_path_with_name, '(.*)\.' ||a_name||'$',subexpression=>1); end; - procedure print_test_results(a_test ut_test) is + function add_test_results(a_test ut_test) return ut_varchar2_rows is l_results ut_varchar2_rows := ut_varchar2_rows(); begin - self.print_text(''); + /* According to specs : - A failure is a test which the code has explicitly failed by using the mechanisms for that purpose. @@ -83,64 +84,83 @@ create or replace type body ut_tfs_junit_reporter is ut_utils.append_to_list( l_results, ''); - self.print_text_lines(l_results); + return l_results; end; - procedure print_suite_results(a_suite ut_logical_suite, a_suite_id in out nocopy integer) is + procedure print_suite_results(a_suite ut_logical_suite, a_suite_id in out nocopy integer, a_nested_tests in out nocopy ut_varchar2_rows) is l_tests_count integer := a_suite.results_count.disabled_count + a_suite.results_count.success_count + a_suite.results_count.failure_count + a_suite.results_count.errored_count; l_results ut_varchar2_rows := ut_varchar2_rows(); l_suite ut_suite; l_outputs clob; l_errors ut_varchar2_list; - begin - + l_tests ut_varchar2_list; + begin for i in 1 .. a_suite.items.count loop if a_suite.items(i) is of(ut_logical_suite) then - print_suite_results(treat(a_suite.items(i) as ut_logical_suite), a_suite_id); + print_suite_results(treat(a_suite.items(i) as ut_logical_suite), a_suite_id, a_nested_tests); end if; end loop; - - if a_suite is of(ut_suite) then - a_suite_id := a_suite_id + 1; - self.print_text(''); - self.print_text(''); + --Due to fact tha TFS and junit5 accepts only flat structure we have to report in suite level only. + if a_suite.self_type ='UT_SUITE_CONTEXT' then for i in 1 .. a_suite.items.count loop if a_suite.items(i) is of(ut_test) then - print_test_results(treat(a_suite.items(i) as ut_test)); + ut_utils.append_to_list( a_nested_tests,(add_test_results(treat(a_suite.items(i) as ut_test)))); end if; end loop; - l_suite := treat(a_suite as ut_suite); - l_outputs := l_suite.get_serveroutputs(); - if l_outputs is not null and l_outputs != empty_clob() then - ut_utils.append_to_list( l_results, ''); - ut_utils.append_to_list( l_results, ut_utils.to_cdata( l_suite.get_serveroutputs() ) ); - ut_utils.append_to_list( l_results, ''); - else - ut_utils.append_to_list( l_results, ''); - end if; - - l_errors := l_suite.get_error_stack_traces(); - if l_errors is not empty then - ut_utils.append_to_list( l_results, ''); - ut_utils.append_to_list( l_results, ut_utils.to_cdata( ut_utils.convert_collection( l_errors ) ) ); - ut_utils.append_to_list( l_results, ''); - else - ut_utils.append_to_list( l_results, ''); - end if; - ut_utils.append_to_list( l_results, ''); - - self.print_text_lines(l_results); + elsif a_suite.self_type ='UT_SUITE' then + for i in 1 .. a_suite.items.count loop + if a_suite.items(i) is of(ut_test) then + ut_utils.append_to_list( a_nested_tests,(add_test_results(treat(a_suite.items(i) as ut_test)))); + end if; + end loop; + --TFS doesnt report on empty test suites, however all we want to make sure is that we dont pring parents suites + -- showing test count but not tests. + if (a_nested_tests.count > 0 and l_tests_count > 0) or (a_nested_tests.count = 0 and l_tests_count = 0) then + a_suite_id := a_suite_id + 1; + ut_utils.append_to_list( l_results,''); + ut_utils.append_to_list( l_results,''); + ut_utils.append_to_list(l_results,a_nested_tests); + l_suite := treat(a_suite as ut_suite); + l_outputs := l_suite.get_serveroutputs(); + if l_outputs is not null and l_outputs != empty_clob() then + ut_utils.append_to_list( l_results, ''); + ut_utils.append_to_list( l_results, ut_utils.to_cdata( l_suite.get_serveroutputs() ) ); + ut_utils.append_to_list( l_results, ''); + else + ut_utils.append_to_list( l_results, ''); + end if; + + l_errors := l_suite.get_error_stack_traces(); + if l_errors is not empty then + ut_utils.append_to_list( l_results, ''); + ut_utils.append_to_list( l_results, ut_utils.to_cdata( ut_utils.convert_collection( l_errors ) ) ); + ut_utils.append_to_list( l_results, ''); + else + ut_utils.append_to_list( l_results, ''); + end if; + ut_utils.append_to_list( l_results, ''); + + self.print_text_lines(l_results); + --We have resolved a context and we now reset value. + a_nested_tests := ut_varchar2_rows(); + end if; end if; end; + + procedure get_suite_results(a_suite ut_logical_suite, a_suite_id in out nocopy integer) is + l_nested_tests ut_varchar2_rows:= ut_varchar2_rows(); + begin + print_suite_results(a_suite, l_suite_id,l_nested_tests); + end; begin l_suite_id := 0; self.print_text(ut_utils.get_xml_header(a_run.client_character_set)); self.print_text(''); for i in 1 .. a_run.items.count loop - print_suite_results(treat(a_run.items(i) as ut_logical_suite), l_suite_id); + get_suite_results(treat(a_run.items(i) as ut_logical_suite), l_suite_id); end loop; self.print_text(''); end; diff --git a/test/ut3_user/reporters/test_tfs_junit_reporter.pkb b/test/ut3_user/reporters/test_tfs_junit_reporter.pkb index 7eb4f8ab6..1036be2ba 100644 --- a/test/ut3_user/reporters/test_tfs_junit_reporter.pkb +++ b/test/ut3_user/reporters/test_tfs_junit_reporter.pkb @@ -8,7 +8,7 @@ create or replace package body test_tfs_junit_reporter as --%test(A test with ) procedure test_do_stuff; - + end;]'; execute immediate q'[create or replace package body check_junit_reporting is procedure test_do_stuff is @@ -18,12 +18,12 @@ create or replace package body test_tfs_junit_reporter as end; end;]'; - + execute immediate q'[create or replace package check_junit_rep_suitepath is --%suitepath(core) --%suite(check_junit_rep_suitepath) --%displayname(Check JUNIT Get path for suitepath) - + --%test(check_junit_rep_suitepath) --%displayname(Check JUNIT Get path for suitepath) procedure check_junit_rep_suitepath; @@ -38,7 +38,7 @@ create or replace package body test_tfs_junit_reporter as execute immediate q'[create or replace package check_junit_flat_suitepath is --%suitepath(core.check_junit_rep_suitepath) --%suite(flatsuitepath) - + --%beforeall procedure donuffin; end;]'; @@ -49,8 +49,85 @@ create or replace package body test_tfs_junit_reporter as end; end;]'; - end; + execute immediate q'[create or replace package check_junit_in_context is + --%suitepath(core.check_junit_rep_suitepath) + --%suite(inctxsuite) + --%displayname(JUNIT test are inside context) + + -- %context(incontext) + -- %name(incontext) + + --%test(incontext) + --%displayname(Check JUNIT Get path incontext) + procedure check_junit_rep_incontext; + + -- %endcontext + end;]'; + execute immediate q'[create or replace package body check_junit_in_context is + procedure check_junit_rep_incontext is + begin + ut3_develop.ut.expect(1).to_equal(1); + end; + end;]'; + + execute immediate q'[create or replace package check_junit_out_context is + --%suitepath(core) + --%suite(outctxsuite) + --%displayname(JUNIT test are outside context) + + -- %context(outcontext) + -- %name(outcontext) + + -- %endcontext + + + --%test(outctx) + --%displayname(outctx) + procedure outctx; + + + end;]'; + execute immediate q'[create or replace package body check_junit_out_context is + procedure outctx is + begin + ut3_develop.ut.expect(1).to_equal(1); + end; + end;]'; + + execute immediate q'[create or replace package check_junit_inout_context is + --%suitepath(core) + --%suite(inoutcontext) + --%displayname(Test in and out of context) + -- %context(incontext) + -- %name(ProductincontextFeatures) + + --%test(inctx) + --%displayname(inctx) + procedure inctx; + + -- %endcontext + + + --%test(outctx) + --%displayname(outctx) + procedure outctx; + + + end;]'; + execute immediate q'[create or replace package body check_junit_inout_context is + procedure inctx is + begin + ut3_develop.ut.expect(1).to_equal(1); + end; + + procedure outctx is + begin + ut3_develop.ut.expect(1).to_equal(1); + end; + end;]'; + + end; procedure escapes_special_chars is l_results ut3_develop.ut_varchar2_list; @@ -92,7 +169,7 @@ create or replace package body test_tfs_junit_reporter as --Assert ut.expect(l_actual).to_be_like('%testcase classname="check_junit_reporting"%'); end; - + procedure check_flatten_nested_suites is l_results ut3_develop.ut_varchar2_list; l_actual clob; @@ -111,7 +188,7 @@ create or replace package body test_tfs_junit_reporter as %'); end; - + procedure check_nls_number_formatting is l_results ut3_develop.ut_varchar2_list; l_actual clob; @@ -163,5 +240,73 @@ create or replace package body test_tfs_junit_reporter as reporters.check_xml_encoding_included(ut3_develop.ut_tfs_junit_reporter(), 'UTF-8'); end; + procedure reports_only_test_in_ctx is + l_results ut3_develop.ut_varchar2_list; + l_actual clob; + begin + --Act + select * + bulk collect into l_results + from table(ut3_develop.ut.run('check_junit_in_context',ut3_develop.ut_tfs_junit_reporter())); + l_actual := ut3_tester_helper.main_helper.table_to_clob(l_results); + --Assert + ut.expect(l_actual).to_be_like(' + + + + + + + + +%'); + end; + + procedure reports_only_test_out_ctx is + l_results ut3_develop.ut_varchar2_list; + l_actual clob; + begin + --Act + select * + bulk collect into l_results + from table(ut3_develop.ut.run('check_junit_out_context',ut3_develop.ut_tfs_junit_reporter())); + l_actual := ut3_tester_helper.main_helper.table_to_clob(l_results); + --Assert + ut.expect(l_actual).to_be_like(' + + + + + + + + +%'); + end; + + procedure reports_only_test_inout_ctx is + l_results ut3_develop.ut_varchar2_list; + l_actual clob; + begin + --Act + select * + bulk collect into l_results + from table(ut3_develop.ut.run('check_junit_inout_context',ut3_develop.ut_tfs_junit_reporter())); + l_actual := ut3_tester_helper.main_helper.table_to_clob(l_results); + --Assert + ut.expect(l_actual).to_be_like(' + + + + + + + + + + +%'); + end; + end; -/ +/ \ No newline at end of file diff --git a/test/ut3_user/reporters/test_tfs_junit_reporter.pks b/test/ut3_user/reporters/test_tfs_junit_reporter.pks index 07acb4d21..70f78e334 100644 --- a/test/ut3_user/reporters/test_tfs_junit_reporter.pks +++ b/test/ut3_user/reporters/test_tfs_junit_reporter.pks @@ -30,6 +30,15 @@ create or replace package test_tfs_junit_reporter as --%test(Includes XML header with encoding when encoding provided) procedure check_encoding_included; + --%test(Reports only testsuites where there are any testcases, all tests are in context) + procedure reports_only_test_in_ctx; + + --%test(Reports only testsuites where there are any testcases, all tests are outside context) + procedure reports_only_test_out_ctx; + + --%test(Reports only testsuites where there are any testcases, one test in ctx one test outside) + procedure reports_only_test_inout_ctx; + --%afterall procedure remove_test_package; end; From d974ce31394118fbb6c1ca9d6aac5a290e16853d Mon Sep 17 00:00:00 2001 From: Lukasz Wasylow Date: Fri, 29 Sep 2023 19:42:46 +0100 Subject: [PATCH 72/77] Update code --- source/reporters/ut_tfs_junit_reporter.tpb | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/source/reporters/ut_tfs_junit_reporter.tpb b/source/reporters/ut_tfs_junit_reporter.tpb index 5b8702808..5e36aad17 100644 --- a/source/reporters/ut_tfs_junit_reporter.tpb +++ b/source/reporters/ut_tfs_junit_reporter.tpb @@ -97,18 +97,22 @@ create or replace type body ut_tfs_junit_reporter is l_tests ut_varchar2_list; begin for i in 1 .. a_suite.items.count loop - if a_suite.items(i) is of(ut_logical_suite) then - print_suite_results(treat(a_suite.items(i) as ut_logical_suite), a_suite_id, a_nested_tests); + if a_suite.items(i) is of(ut_suite_context) then + print_suite_results(treat(a_suite.items(i) as ut_suite_context), a_suite_id, a_nested_tests); + elsif a_suite.items(i) is of(ut_suite) then + print_suite_results(treat(a_suite.items(i) as ut_suite), a_suite_id, a_nested_tests); + elsif a_suite.items(i) is of(ut_logical_suite) then + print_suite_results(treat(a_suite.items(i) as ut_logical_suite), a_suite_id, a_nested_tests); end if; end loop; --Due to fact tha TFS and junit5 accepts only flat structure we have to report in suite level only. - if a_suite.self_type ='UT_SUITE_CONTEXT' then + if a_suite is of(ut_suite_context) then for i in 1 .. a_suite.items.count loop if a_suite.items(i) is of(ut_test) then ut_utils.append_to_list( a_nested_tests,(add_test_results(treat(a_suite.items(i) as ut_test)))); end if; end loop; - elsif a_suite.self_type ='UT_SUITE' then + elsif a_suite is of(ut_suite) then for i in 1 .. a_suite.items.count loop if a_suite.items(i) is of(ut_test) then ut_utils.append_to_list( a_nested_tests,(add_test_results(treat(a_suite.items(i) as ut_test)))); From 03b0566bda57947df59eb0dd3bfbdb203a6a79c4 Mon Sep 17 00:00:00 2001 From: Rachid EL AISSAOUI Date: Tue, 5 Dec 2023 23:46:45 +0100 Subject: [PATCH 73/77] Fix indentation --- docs/userguide/best-practices.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/userguide/best-practices.md b/docs/userguide/best-practices.md index 5148273e6..57a108fdd 100644 --- a/docs/userguide/best-practices.md +++ b/docs/userguide/best-practices.md @@ -15,9 +15,9 @@ The following are best practices we at utPLSQL have learned about PL/SQL and Uni - Tests should not mimic / duplicate the logic of tested code - Tests should contain zero logic (or as close to zero as possible) - The 3A rule: - - Arrange (setup inputs/data/environment for the tested code) - - Act (execute code under test) - - Assert (validate the outcomes of the execution) + - Arrange (setup inputs/data/environment for the tested code) + - Act (execute code under test) + - Assert (validate the outcomes of the execution) - Each tested procedure/function/trigger (code block) should have more than one test - Each test should check only one behavior (one requirement) of the code block under test - Tests should be maintained as thoroughly as production code From 03c231cd44b60f218dd61c2f8a1abf8713d3503d Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" Date: Sun, 18 Feb 2024 15:58:08 +0000 Subject: [PATCH 74/77] Updated project version after build [skip ci] --- docs/about/authors.md | 2 +- docs/about/license.md | 2 +- docs/about/project-details.md | 2 +- docs/about/support.md | 2 +- docs/index.md | 2 +- docs/userguide/advanced_data_comparison.md | 2 +- docs/userguide/annotations.md | 2 +- docs/userguide/best-practices.md | 2 +- docs/userguide/coverage.md | 2 +- docs/userguide/exception-reporting.md | 2 +- docs/userguide/expectations.md | 2 +- docs/userguide/getting-started.md | 2 +- docs/userguide/install.md | 2 +- docs/userguide/querying_suites.md | 2 +- docs/userguide/reporters.md | 2 +- docs/userguide/running-unit-tests.md | 2 +- docs/userguide/upgrade.md | 2 +- source/core/ut_utils.pks | 2 +- 18 files changed, 18 insertions(+), 18 deletions(-) diff --git a/docs/about/authors.md b/docs/about/authors.md index c4c40bfad..228cb8451 100644 --- a/docs/about/authors.md +++ b/docs/about/authors.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4187--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4194--develop-blue.svg) ### utPLSQL v3 Major Contributors diff --git a/docs/about/license.md b/docs/about/license.md index 702ea3f9c..fc4dbea8b 100644 --- a/docs/about/license.md +++ b/docs/about/license.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4187--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4194--develop-blue.svg) # Version Information diff --git a/docs/about/project-details.md b/docs/about/project-details.md index 60aa6a657..cb30adf37 100644 --- a/docs/about/project-details.md +++ b/docs/about/project-details.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4187--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4194--develop-blue.svg) # utPLSQL Project Details diff --git a/docs/about/support.md b/docs/about/support.md index fc412205d..452c69dab 100644 --- a/docs/about/support.md +++ b/docs/about/support.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4187--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4194--develop-blue.svg) # How to get support diff --git a/docs/index.md b/docs/index.md index 8325ec636..01a850083 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4187--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4194--develop-blue.svg) ## What is utPLSQL diff --git a/docs/userguide/advanced_data_comparison.md b/docs/userguide/advanced_data_comparison.md index 1f7f2d191..df8c9d47d 100644 --- a/docs/userguide/advanced_data_comparison.md +++ b/docs/userguide/advanced_data_comparison.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4187--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4194--develop-blue.svg) # Advanced data comparison diff --git a/docs/userguide/annotations.md b/docs/userguide/annotations.md index d1ff5da07..72c70de2e 100644 --- a/docs/userguide/annotations.md +++ b/docs/userguide/annotations.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4187--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4194--develop-blue.svg) Annotations are used to configure tests and suites in a declarative way similar to modern OOP languages. This way, test configuration is stored along with the test logic inside the test package. No additional configuration files or tables are needed for test cases. The annotation names are based on popular testing frameworks such as JUnit. diff --git a/docs/userguide/best-practices.md b/docs/userguide/best-practices.md index 57a108fdd..d1448fa23 100644 --- a/docs/userguide/best-practices.md +++ b/docs/userguide/best-practices.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4187--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4194--develop-blue.svg) The following are best practices we at utPLSQL have learned about PL/SQL and Unit Testing. diff --git a/docs/userguide/coverage.md b/docs/userguide/coverage.md index 5d071335e..d3e7ef42e 100644 --- a/docs/userguide/coverage.md +++ b/docs/userguide/coverage.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4187--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4194--develop-blue.svg) utPLSQL comes with a built-in coverage reporting engine. The code coverage reporting uses package DBMS_PROFILER (and DBMS_PLSQL_CODE_COVERAGE on Oracle database version 12.2 and above) provided with Oracle database. Code coverage is gathered for the following source types: diff --git a/docs/userguide/exception-reporting.md b/docs/userguide/exception-reporting.md index ca52ac951..43d67eb68 100644 --- a/docs/userguide/exception-reporting.md +++ b/docs/userguide/exception-reporting.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4187--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4194--develop-blue.svg) utPLSQL is responsible for handling exceptions wherever they occur in the test run. The framework is trapping most of the exceptions so that the test execution is not affected by individual tests or test packages throwing an exception. The framework provides a full stacktrace for every exception that was thrown. The reported stacktrace does not include any utPLSQL library calls in it. diff --git a/docs/userguide/expectations.md b/docs/userguide/expectations.md index b6f3d7e29..c4ddb4757 100644 --- a/docs/userguide/expectations.md +++ b/docs/userguide/expectations.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4187--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4194--develop-blue.svg) ## Expectation concepts diff --git a/docs/userguide/getting-started.md b/docs/userguide/getting-started.md index 9620878ad..964ccbdcf 100644 --- a/docs/userguide/getting-started.md +++ b/docs/userguide/getting-started.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4187--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4194--develop-blue.svg) # Getting started with TDD and utPLSQL diff --git a/docs/userguide/install.md b/docs/userguide/install.md index 8f32c5a10..08bd036b0 100644 --- a/docs/userguide/install.md +++ b/docs/userguide/install.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4187--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4194--develop-blue.svg) ## Supported database versions diff --git a/docs/userguide/querying_suites.md b/docs/userguide/querying_suites.md index e3ec1f53d..77307bf7c 100644 --- a/docs/userguide/querying_suites.md +++ b/docs/userguide/querying_suites.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4187--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4194--develop-blue.svg) ## Obtaining information about suites diff --git a/docs/userguide/reporters.md b/docs/userguide/reporters.md index e37ac6627..cd49d7c38 100644 --- a/docs/userguide/reporters.md +++ b/docs/userguide/reporters.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4187--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4194--develop-blue.svg) utPLSQL provides several reporting formats. The sections below describe most of them. diff --git a/docs/userguide/running-unit-tests.md b/docs/userguide/running-unit-tests.md index 3b23f5f98..941a00e36 100644 --- a/docs/userguide/running-unit-tests.md +++ b/docs/userguide/running-unit-tests.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4187--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4194--develop-blue.svg) utPLSQL framework provides two main entry points to run unit tests from within the database: diff --git a/docs/userguide/upgrade.md b/docs/userguide/upgrade.md index e5537f520..623bef999 100644 --- a/docs/userguide/upgrade.md +++ b/docs/userguide/upgrade.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4187--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4194--develop-blue.svg) # Upgrading from version 2 diff --git a/source/core/ut_utils.pks b/source/core/ut_utils.pks index 522da1480..9bdf0c7a5 100644 --- a/source/core/ut_utils.pks +++ b/source/core/ut_utils.pks @@ -21,7 +21,7 @@ create or replace package ut_utils authid definer is * */ - gc_version constant varchar2(50) := 'v3.1.14.4187-develop'; + gc_version constant varchar2(50) := 'v3.1.14.4194-develop'; subtype t_executable_type is varchar2(30); gc_before_all constant t_executable_type := 'beforeall'; From 688aad2f0a9d4b39b69d44de5e7d0b5834966702 Mon Sep 17 00:00:00 2001 From: Jacek Gebal Date: Mon, 19 Feb 2024 23:14:30 +0200 Subject: [PATCH 75/77] Simplification of the release process. --- .github/scripts/get_project_build_version.sh | 2 -- .github/scripts/get_project_version.sh | 13 +++-------- .../set_release_version_numbers_env.sh | 9 ++++++++ .github/scripts/set_version_numbers_env.sh | 12 ++++------ .github/scripts/update_project_version.sh | 1 + .github/workflows/build.yml | 2 +- .github/workflows/release.yml | 2 +- BUILD_NO | 1 + development/releasing.md | 23 ++++++++----------- 9 files changed, 31 insertions(+), 34 deletions(-) delete mode 100755 .github/scripts/get_project_build_version.sh create mode 100755 .github/scripts/set_release_version_numbers_env.sh create mode 100644 BUILD_NO diff --git a/.github/scripts/get_project_build_version.sh b/.github/scripts/get_project_build_version.sh deleted file mode 100755 index 838523b05..000000000 --- a/.github/scripts/get_project_build_version.sh +++ /dev/null @@ -1,2 +0,0 @@ -#!/usr/bin/env bash -echo `sed -E "s/(v?[0-9]+\.)([0-9]+\.)([0-9]+)(-.*)?/\1\2\3\.${UTPLSQL_BUILD_NO}\4/" <<< "${UTPLSQL_VERSION}"` diff --git a/.github/scripts/get_project_version.sh b/.github/scripts/get_project_version.sh index 60fc0a796..54a30a562 100755 --- a/.github/scripts/get_project_version.sh +++ b/.github/scripts/get_project_version.sh @@ -1,14 +1,7 @@ #!/usr/bin/env bash - -#When building a new version from a release branch, the version is taken from release branch name -if [[ "${CI_ACTION_REF_NAME}" =~ ^release/v[0-9]+\.[0-9]+\.[0-9]+.*$ ]]; then - version=${CI_ACTION_REF_NAME#release\/} -else - #Otherwise, version is taken from the VERSION file - version=`cat VERSION` - #When on develop branch, add "-develop" to the version text - if [[ "${CI_ACTION_REF_NAME}" == "develop" ]]; then +version=`cat VERSION` +#When on develop branch, add "-develop" to the version text +if [[ "${CI_ACTION_REF_NAME}" == "develop" ]]; then version=`sed -E "s/(v?[0-9]+\.[0-9]+\.[0-9]+).*/\1-develop/" <<< "${version}"` - fi fi echo ${version} diff --git a/.github/scripts/set_release_version_numbers_env.sh b/.github/scripts/set_release_version_numbers_env.sh new file mode 100755 index 000000000..a43843508 --- /dev/null +++ b/.github/scripts/set_release_version_numbers_env.sh @@ -0,0 +1,9 @@ +#!/bin/bash + +build_no=$(cat BUILD_NO) +version=${CI_ACTION_REF_NAME} + +echo "UTPLSQL_BUILD_NO=${build_no}" >> $GITHUB_ENV +echo "UTPLSQL_VERSION=${version}" >> $GITHUB_ENV +echo UTPLSQL_BUILD_VERSION=$(echo ${version} | sed -E "s/(v?[0-9]+\.)([0-9]+\.)([0-9]+)(-.*)?/\1\2\3\.${build_no}\4/") >> $GITHUB_ENV + diff --git a/.github/scripts/set_version_numbers_env.sh b/.github/scripts/set_version_numbers_env.sh index 10529a1bf..c61639bf9 100755 --- a/.github/scripts/set_version_numbers_env.sh +++ b/.github/scripts/set_version_numbers_env.sh @@ -1,10 +1,8 @@ #!/bin/bash -UTPLSQL_BUILD_NO=$( expr ${GITHUB_RUN_NUMBER} + ${UTPLSQL_BUILD_NO_OFFSET} ) -UTPLSQL_VERSION=$(.github/scripts/get_project_version.sh) +build_no=$( expr ${GITHUB_RUN_NUMBER} + ${UTPLSQL_BUILD_NO_OFFSET} ) +version=$(.github/scripts/get_project_version.sh) -echo "UTPLSQL_BUILD_NO=${UTPLSQL_BUILD_NO}" >> $GITHUB_ENV -echo "UTPLSQL_VERSION=${UTPLSQL_VERSION}" >> $GITHUB_ENV -echo UTPLSQL_BUILD_VERSION=$(echo ${UTPLSQL_VERSION} | sed -E "s/(v?[0-9]+\.)([0-9]+\.)([0-9]+)(-.*)?/\1\2\3\.${UTPLSQL_BUILD_NO}\4/") >> $GITHUB_ENV - -echo "CURRENT_BRANCH=${CI_ACTION_REF_NAME}" >> $GITHUB_ENV +echo "UTPLSQL_BUILD_NO=${build_no}" >> $GITHUB_ENV +echo "UTPLSQL_VERSION=${version}" >> $GITHUB_ENV +echo UTPLSQL_BUILD_VERSION=$(echo ${version} | sed -E "s/(v?[0-9]+\.)([0-9]+\.)([0-9]+)(-.*)?/\1\2\3\.${build_no}\4/") >> $GITHUB_ENV diff --git a/.github/scripts/update_project_version.sh b/.github/scripts/update_project_version.sh index 586cb5b64..66e45125a 100755 --- a/.github/scripts/update_project_version.sh +++ b/.github/scripts/update_project_version.sh @@ -16,4 +16,5 @@ sed -i -r "s/(sonar\.projectVersion=).*?/\1${UTPLSQL_VERSION}/" sonar-project.pr echo Update VERSION file echo ${UTPLSQL_VERSION} > VERSION +echo ${UTPLSQL_BUILD_NO} > BUILD_NO diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index c2fe3971e..668f1e5a4 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -245,7 +245,7 @@ jobs: - name: Push version update to repository id: push-version-number-update run: | - git add sonar-project.properties VERSION source/* docs/* + git add sonar-project.properties VERSION BUILD_NO source/* docs/* git commit -m 'Updated project version after build [skip ci]' git push --quiet origin HEAD:${CI_ACTION_REF_NAME} diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index a748d1cbd..0e29bd5c1 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -25,7 +25,7 @@ jobs: - uses: FranzDiebold/github-env-vars-action@v2 #https://github.com/marketplace/actions/github-environment-variables-action - name: Set build version number env variables - run: .github/scripts/set_version_numbers_env.sh + run: .github/scripts/set_release_version_numbers_env.sh - name: Update project version & build number in source code and documentation run: .github/scripts/update_project_version.sh diff --git a/BUILD_NO b/BUILD_NO new file mode 100644 index 000000000..9ae5f9c77 --- /dev/null +++ b/BUILD_NO @@ -0,0 +1 @@ +4194 \ No newline at end of file diff --git a/development/releasing.md b/development/releasing.md index 2b9be317a..c1e49184d 100644 --- a/development/releasing.md +++ b/development/releasing.md @@ -3,28 +3,25 @@ To create a release follow the below steps ## Release preparation - - Create a **draft** of a Release with version number `vX.Y.X` sourced from the `main` branch using [github releases page](https://github.com/utPLSQL/utPLSQL/releases) and populate release description using information found on the issues and pull requests **since previous release**. - To find issues closed after certain date use [advanced filters](https://help.github.com/articles/searching-issues-and-pull-requests/#search-by-open-or-closed-state). - Example: [`is:issue closed:>2018-07-22`](https://github.com/utPLSQL/utPLSQL/issues?utf8=%E2%9C%93&q=is%3Aissue+closed%3A%3E2018-07-22+) + - Create a **draft** of a Release with a new tag number `vX.Y.X` sourced from the `develop` branch on [github releases page](https://github.com/utPLSQL/utPLSQL/releases) + - Populate release description using the `Generate release notes` button + - Review the auto-generated release notes and update tem if needed + - Split the default `## What's Changed` list into `## New features`, `## Enhancements`, `## Bug fixes`. See previous release notes for details ## Performing a release - - create the release branch from `develop` branch and make sure to name the release branch: `release/vX.Y.Z` - - update, commit and push at least one file change in the release branch, to kick off a build on [GithubActions](https://github.com/utPLSQL/utPLSQL/actions) or kick-off a build manually for that branch after it was created on github. - - wait for the build to complete successfully as it will update the version to be release number (without develop) - - merge the release branch to `main` branch and publish [the previously prepared](#release-preparation) release draft. + - Publish [the previously prepared](#release-preparation) release draft. - Wait for the [Github Actions `Release`](https://github.com/utPLSQL/utPLSQL/actions/workflows/release.yml) process to complete successfully. The process will upload release artifacts (`zip` and `tar.gz` files along with `md5`) - - After Release build was completed successfully, merge the `main` branch back into `develop` branch. At this point, main branch and release tag should be at the same commit version and artifacts should be uploaded into Github release. - - After develop branch was built, increase the version number in `VERSION` file to represent next planned release version. + - After Release build was completed successfully, merge the `develop` branch into `main` branch. At this point, main branch and release tag should be at the same commit version and artifacts should be uploaded into Github release. + - Increase the version number in the `VERSION` file on `develop` branch to open start next release version. - Clone `utplsql.githug.io` project and: - Add a new announcement about next version being released in `docs/_posts`. Use previous announcements as a template. Make sure to set date, time and post title properly. - Add the post to list in `mkdocs.yml` file in root directory of that repository. - Add the link to the post at the beginning of the `docs/index.md` file. + - Send the announcement on Twitter(X) accoiunt abut utPLSQL release. The following will happen: - - build executed on branch `release/vX.Y.Z-[something]` updates files `sonar-project.properties`, `VERSION` with project version derived from the release branch name - - changes to those two files are committed and pushed back to release branch - - when a Github release is published, a new tag is added in on the repository and a release build is executed - - With Release build, the documentation for new release is published on `utplsql.github.io` and installation archives are added to the release. + - When a Github release is published, a new tag is added in on the repository and a release build is executed + - With Release action, the documentation for new release is published on `utplsql.github.io` and installation archives are added to the release. # Note: The utPLSQL installation files are uploaded by the release build process as release artifacts (separate `zip` and `tar.gz` files). From fc9382e27515762ca5a4cbfa747d54de47e2675c Mon Sep 17 00:00:00 2001 From: Jacek Gebal Date: Mon, 19 Feb 2024 23:24:21 +0200 Subject: [PATCH 76/77] Fixed a flaky test when test runs in 0 seconds. --- test/ut3_user/api/test_ut_run.pkb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/test/ut3_user/api/test_ut_run.pkb b/test/ut3_user/api/test_ut_run.pkb index e80235744..f0cfd8373 100644 --- a/test/ut3_user/api/test_ut_run.pkb +++ b/test/ut3_user/api/test_ut_run.pkb @@ -742,7 +742,7 @@ Failures:% procedure remove_time_from_results(a_results in out nocopy ut3_develop.ut_varchar2_list) is begin for i in 1 .. a_results.count loop - a_results(i) := regexp_replace(a_results(i),'\[[0-9]*[\.,][0-9]+ sec\]',''); + a_results(i) := regexp_replace(a_results(i),'\[[0-9]*[\.,]?[0-9]+ sec\]',''); a_results(i) := regexp_replace(a_results(i),'Finished in [0-9]*[\.,][0-9]+ seconds',''); end loop; end; From 072dfbc1d0ed37f8da9c0ff87823906483ea3887 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" Date: Mon, 19 Feb 2024 21:41:43 +0000 Subject: [PATCH 77/77] Updated project version after build [skip ci] --- BUILD_NO | 2 +- docs/about/authors.md | 2 +- docs/about/license.md | 2 +- docs/about/project-details.md | 2 +- docs/about/support.md | 2 +- docs/index.md | 2 +- docs/userguide/advanced_data_comparison.md | 2 +- docs/userguide/annotations.md | 2 +- docs/userguide/best-practices.md | 2 +- docs/userguide/coverage.md | 2 +- docs/userguide/exception-reporting.md | 2 +- docs/userguide/expectations.md | 2 +- docs/userguide/getting-started.md | 2 +- docs/userguide/install.md | 2 +- docs/userguide/querying_suites.md | 2 +- docs/userguide/reporters.md | 2 +- docs/userguide/running-unit-tests.md | 2 +- docs/userguide/upgrade.md | 2 +- source/core/ut_utils.pks | 2 +- 19 files changed, 19 insertions(+), 19 deletions(-) diff --git a/BUILD_NO b/BUILD_NO index 9ae5f9c77..26de2ed80 100644 --- a/BUILD_NO +++ b/BUILD_NO @@ -1 +1 @@ -4194 \ No newline at end of file +4197 diff --git a/docs/about/authors.md b/docs/about/authors.md index 228cb8451..203d450d1 100644 --- a/docs/about/authors.md +++ b/docs/about/authors.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4194--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4197--develop-blue.svg) ### utPLSQL v3 Major Contributors diff --git a/docs/about/license.md b/docs/about/license.md index fc4dbea8b..28aea5fed 100644 --- a/docs/about/license.md +++ b/docs/about/license.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4194--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4197--develop-blue.svg) # Version Information diff --git a/docs/about/project-details.md b/docs/about/project-details.md index cb30adf37..8e6730847 100644 --- a/docs/about/project-details.md +++ b/docs/about/project-details.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4194--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4197--develop-blue.svg) # utPLSQL Project Details diff --git a/docs/about/support.md b/docs/about/support.md index 452c69dab..0748fd479 100644 --- a/docs/about/support.md +++ b/docs/about/support.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4194--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4197--develop-blue.svg) # How to get support diff --git a/docs/index.md b/docs/index.md index 01a850083..320784892 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4194--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4197--develop-blue.svg) ## What is utPLSQL diff --git a/docs/userguide/advanced_data_comparison.md b/docs/userguide/advanced_data_comparison.md index df8c9d47d..34b4eddcc 100644 --- a/docs/userguide/advanced_data_comparison.md +++ b/docs/userguide/advanced_data_comparison.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4194--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4197--develop-blue.svg) # Advanced data comparison diff --git a/docs/userguide/annotations.md b/docs/userguide/annotations.md index 72c70de2e..b3602ef10 100644 --- a/docs/userguide/annotations.md +++ b/docs/userguide/annotations.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4194--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4197--develop-blue.svg) Annotations are used to configure tests and suites in a declarative way similar to modern OOP languages. This way, test configuration is stored along with the test logic inside the test package. No additional configuration files or tables are needed for test cases. The annotation names are based on popular testing frameworks such as JUnit. diff --git a/docs/userguide/best-practices.md b/docs/userguide/best-practices.md index d1448fa23..379024e73 100644 --- a/docs/userguide/best-practices.md +++ b/docs/userguide/best-practices.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4194--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4197--develop-blue.svg) The following are best practices we at utPLSQL have learned about PL/SQL and Unit Testing. diff --git a/docs/userguide/coverage.md b/docs/userguide/coverage.md index d3e7ef42e..24855cbb5 100644 --- a/docs/userguide/coverage.md +++ b/docs/userguide/coverage.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4194--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4197--develop-blue.svg) utPLSQL comes with a built-in coverage reporting engine. The code coverage reporting uses package DBMS_PROFILER (and DBMS_PLSQL_CODE_COVERAGE on Oracle database version 12.2 and above) provided with Oracle database. Code coverage is gathered for the following source types: diff --git a/docs/userguide/exception-reporting.md b/docs/userguide/exception-reporting.md index 43d67eb68..e2a8bc7da 100644 --- a/docs/userguide/exception-reporting.md +++ b/docs/userguide/exception-reporting.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4194--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4197--develop-blue.svg) utPLSQL is responsible for handling exceptions wherever they occur in the test run. The framework is trapping most of the exceptions so that the test execution is not affected by individual tests or test packages throwing an exception. The framework provides a full stacktrace for every exception that was thrown. The reported stacktrace does not include any utPLSQL library calls in it. diff --git a/docs/userguide/expectations.md b/docs/userguide/expectations.md index c4ddb4757..472c9654e 100644 --- a/docs/userguide/expectations.md +++ b/docs/userguide/expectations.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4194--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4197--develop-blue.svg) ## Expectation concepts diff --git a/docs/userguide/getting-started.md b/docs/userguide/getting-started.md index 964ccbdcf..9c4d8813a 100644 --- a/docs/userguide/getting-started.md +++ b/docs/userguide/getting-started.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4194--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4197--develop-blue.svg) # Getting started with TDD and utPLSQL diff --git a/docs/userguide/install.md b/docs/userguide/install.md index 08bd036b0..c322639df 100644 --- a/docs/userguide/install.md +++ b/docs/userguide/install.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4194--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4197--develop-blue.svg) ## Supported database versions diff --git a/docs/userguide/querying_suites.md b/docs/userguide/querying_suites.md index 77307bf7c..463bcef97 100644 --- a/docs/userguide/querying_suites.md +++ b/docs/userguide/querying_suites.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4194--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4197--develop-blue.svg) ## Obtaining information about suites diff --git a/docs/userguide/reporters.md b/docs/userguide/reporters.md index cd49d7c38..41468a3a5 100644 --- a/docs/userguide/reporters.md +++ b/docs/userguide/reporters.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4194--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4197--develop-blue.svg) utPLSQL provides several reporting formats. The sections below describe most of them. diff --git a/docs/userguide/running-unit-tests.md b/docs/userguide/running-unit-tests.md index 941a00e36..502bd49bd 100644 --- a/docs/userguide/running-unit-tests.md +++ b/docs/userguide/running-unit-tests.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4194--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4197--develop-blue.svg) utPLSQL framework provides two main entry points to run unit tests from within the database: diff --git a/docs/userguide/upgrade.md b/docs/userguide/upgrade.md index 623bef999..cb49a0424 100644 --- a/docs/userguide/upgrade.md +++ b/docs/userguide/upgrade.md @@ -1,4 +1,4 @@ -![version](https://img.shields.io/badge/version-v3.1.14.4194--develop-blue.svg) +![version](https://img.shields.io/badge/version-v3.1.14.4197--develop-blue.svg) # Upgrading from version 2 diff --git a/source/core/ut_utils.pks b/source/core/ut_utils.pks index 9bdf0c7a5..463aa1604 100644 --- a/source/core/ut_utils.pks +++ b/source/core/ut_utils.pks @@ -21,7 +21,7 @@ create or replace package ut_utils authid definer is * */ - gc_version constant varchar2(50) := 'v3.1.14.4194-develop'; + gc_version constant varchar2(50) := 'v3.1.14.4197-develop'; subtype t_executable_type is varchar2(30); gc_before_all constant t_executable_type := 'beforeall';