From dac17804c953bf3956e85bbea7caa0d9cc1c2122 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jacek=20G=C4=99bal?= Date: Sun, 15 Jan 2017 02:28:15 +0000 Subject: [PATCH 1/2] Updated documentation. --- docs/userguide/annotations.md | 23 +-- docs/userguide/assertions.md | 5 - docs/userguide/expectations.md | 78 +++++++++++ examples/between_string/test_betwnstr.pkg | 18 +-- mkdocs.yml | 2 +- readme.md | 162 +++++++++++++--------- 6 files changed, 197 insertions(+), 91 deletions(-) delete mode 100644 docs/userguide/assertions.md create mode 100644 docs/userguide/expectations.md diff --git a/docs/userguide/annotations.md b/docs/userguide/annotations.md index 7b021bba1..9bb833bfe 100644 --- a/docs/userguide/annotations.md +++ b/docs/userguide/annotations.md @@ -5,8 +5,15 @@ The annotation list is based on moder testing framework such as jUnit 5, RSpec. Annotations allow to configure test infrastructure in a declarative way without anything stored in tables or config files. The framework runner scans the schema for all the suitable annotated packages, automatically configures suites, forms hierarchy from then and executes them. -# Example of annotated package -``` +Annotations are case-insensitive. But it is recommended to use the lower-case standard as described in the documentation. + +Annotation on procedure level must be placed directly before the procedure name. + +Annotation `-- %suite` should be placed at the beginning of package specification. It is not required but highly recommended as a practice. + +# Example of annotated test package + +```sql create or replace package test_pkg is -- %suite(Name of suite) @@ -51,14 +58,14 @@ create or replace package test_pkg is end test_pkg; ``` -#Annotations meaning +#Annotations description -| Annotation |Level| Describtion | +| Annotation |Level| Description | | --- | --- | --- | | `%suite()` | Package | Marks package to be a suite of tests This way all testing packages might be found in a schema. Optional schema discription can by provided, similar to `%displayname` annotation. | -| `%suitepath()` | Package | Similar to java package. The annotation allows logical grouping of suites into hierarcies. | -| `%displayname()` | Package/procedure | Human-familiar describtion of the suite/test. Syntax is based on jUnit annotation: `%displayname(Name of the suite/test)` | -| `%test()` | Procedure | Denotes that a method is a test method. Optional test discription can by provided, similar to `%displayname` annotation. | +| `%suitepath()` | Package | Similar to java package. The annotation allows logical grouping of suites into hierarchies. | +| `%displayname()` | Package/procedure | Human-familiar description of the suite/test. Syntax is based on jUnit annotation: `%displayname(Name of the suite/test)` | +| `%test()` | Procedure | Denotes that a method is a test method. Optional test description can by provided, similar to `%displayname` annotation. | | `%beforeall` | Procedure | Denotes that the annotated procedure should be executed once before all elements of the current suite. | | `%afterall` | Procedure | Denotes that the annotated procedure should be executed once after all elements of the current suite. | | `%beforeeach` | Procedure | Denotes that the annotated procedure should be executed before each `%test` method in the current suite. | @@ -66,4 +73,4 @@ end test_pkg; | `%beforetest()` | Procedure | Denotes that mentioned procedure should be executed before the annotated `%test` procedure. | | `%aftertest()` | Procedure | Denotes that mentioned procedure should be executed after the annotated `%test` procedure. | | `%rollback()` | Package/procedure | Configure transaction control behaviour (type). Supported values: `auto`(default) - rollback to savepoint (before the test/suite setup) is issued after each test/suite teardown; `manual` - rollback is never issued automatically. Property can be overridden for child element (test in suite) | -| `%disable` | Package/procedure | Used to disable a suite or a test | \ No newline at end of file +| `%disable` | Package/procedure | Used to disable a suite or a test | diff --git a/docs/userguide/assertions.md b/docs/userguide/assertions.md deleted file mode 100644 index b37479444..000000000 --- a/docs/userguide/assertions.md +++ /dev/null @@ -1,5 +0,0 @@ -# Assertion - - A test procedure will contain one or more checks to verify the the test performed as expected. These checks are called assertion. utPLSQL provides a robust and extensible assertion library. - - TODO: Finish assertion Documentation \ No newline at end of file diff --git a/docs/userguide/expectations.md b/docs/userguide/expectations.md new file mode 100644 index 000000000..0663e035c --- /dev/null +++ b/docs/userguide/expectations.md @@ -0,0 +1,78 @@ +# Concept of expectation and matcher + +Validation of the code under test (the tested logic of procedure/function etc.) is performed by comparing the actual data against the expected data. +To do that we use concept of expectation and a matcher to perform the check on the data. + +It's best to give an example to get an idea what is what +```sql +begin + ut.expect( 'the tested value' ).to_( equal('the expected value') ); +end; +``` + +Expectation is a set of the expected value(s), actual values(s) and the matcher(s) to run on those values. + +Matcher is defining the comparison operation to be performed on expected and actual values. + +# List of currently build-in matchers +- `match` +- `equal` +- `be_true` +- `be_null` +- `be_not_null` +- `be_like` +- `be_less_than` +- `be_less_or_equal` +- `be_greater_than` +- `be_greater_or_equal` +- `be_false` +- `be_between` + +## match +Allows regexp_like validations to be executed against the following datatypes: +- `clob` +- `varchar2` + +Usage: +```sql + ut.expect( a_actual ).to_( match( a_pattern in varchar2, a_modifiers in varchar2 := null) ) +``` + +Parameters `a_pattern` and `a_modifiers` represent a valid regexp pattern accepted by [Oracle regexp_like function](https://docs.oracle.com/cd/E11882_01/server.112/e41084/conditions007.htm#SQLRF00501) + +## equal + +The equal matcher is a very restrictive matcher. +It only returns true, if compared data-types. +That means, that comparing varchar2 to a number will fail even if the varchar2 contains the same number. +This matcher is designed to capture changes of data-type, so that if you expect your variable to be number and is now something else, + the test will fail and give you early indication of potential problem. + +Usage: +```sql + ut.expect( a_actual ).to_( equal( a_expected {mulitple data-types}, a_nulls_are_equal boolean := null) ) +``` + + +The equal matcher accepts a_expected of following data-types. +- `anydata` +- `blob` +- `boolean` +- `clob` +- `date` +- `number` +- `sys_refcursor` +- `timestamp_unconstrained` +- `timestamp_tz_unconstrained` +- `timestamp_ltz_unconstrained` +- `varchar2` +- `yminterval_unconstrained` +- `dsinterval_unconstrained` + +The second parameter decides on the behavior of `null=null` comparison (**this comparison by default is true!**) + + + A test procedure will contain one or more checks to verify the the test performed as expected. These checks are called assertion. utPLSQL provides a robust and extensible assertion library. + + +TODO: Finish Expectations concepts diff --git a/examples/between_string/test_betwnstr.pkg b/examples/between_string/test_betwnstr.pkg index b55f8f083..34a4545cb 100644 --- a/examples/between_string/test_betwnstr.pkg +++ b/examples/between_string/test_betwnstr.pkg @@ -1,22 +1,17 @@ create or replace package test_betwnstr as - -- %suite - -- %displayname(Between string function) + -- %suite(Between string function) - -- %test - -- %displayname(Returns substring from start position to end position) + -- %test(Returns substring from start position to end position) procedure normal_case; - -- %test - -- %displayname(Returns substring when start position is zero) + -- %test(Returns substring when start position is zero) procedure zero_start_position; - -- %test - -- %displayname(Returns string until end if end position is greated than string length) + -- %test(Returns string until end if end position is greater than string length) procedure big_end_position; - -- %test - -- %displayname(Returns null for null inlut srting value) + -- %test(Returns null for null input string value) procedure null_string; end; / @@ -24,7 +19,7 @@ create or replace package body test_betwnstr as procedure normal_case is begin - ut.expect( betwnstr( '1234567', 2, 5 ) ).to_( equal('2345') ); + ut.expect( betwnstr( '1234567', 2, 5 ) ).to_equal('2345'); end; procedure zero_start_position is @@ -32,7 +27,6 @@ create or replace package body test_betwnstr as ut.expect( betwnstr( '1234567', 0, 5 ) ).to_( equal('12345') ); end; - procedure big_end_position is begin ut.expect( betwnstr( '1234567', 0, 500 ) ).to_( equal('1234567') ); diff --git a/mkdocs.yml b/mkdocs.yml index 40a4e835e..4c7cf5404 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -14,7 +14,7 @@ pages: - Installation: userguide/install.md - Getting Started: userguide/getting-started.md - Annotations: userguide/annotations.md - - Assertions: userguide/assertions.md + - Expectations: userguide/expectations.md - Testing Best Pracitces: userguide/best-practices.md - Upgrade utPLSQL : userguide/upgrade.md - About: diff --git a/readme.md b/readme.md index cf9964a82..09789b82c 100644 --- a/readme.md +++ b/readme.md @@ -6,18 +6,31 @@ [![twitter](https://img.shields.io/twitter/follow/utPLSQL.svg?style=social&label=Follow)](https://twitter.com/utPLSQL) ---------- -Version 3 is a complete rewrite of utPLSQL from scratch. Version 2 still supports older versions of Oracle that are no longer available. This has lead to difficult to maintain code. Yet many developers wanted to take it to the next level. The community that had developed on GitHub, decided that a new internal architecture was needed, from that version 3 was born. Currently version 3 is not complete and is not ready for a production environment as the API is not stable and changing. However it is quickly taking shape. We welcome new developers to join our community and help utPLSQL grow. - -Primary Goals: - - Easier to maintain - - Only supports versions of Oracle under [Extend Support](http://www.oracle.com/us/support/library/lifetime-support-technology-069183.pdf) (Currently 11.2, and 12.1) - - API is documented in the code where possible. - - Robust and vibrant assertion library. - - Support for Code Coverage - - Extensible API - - Published upgrade/conversion path from version 2. - - Easily called from current PL/SQL development tools - - More permissive License to allow vendors easier ability to integrate utPLSQL. +utPLSQL version 3 is a complete rewrite of utPLSQL v2 from scratch. +Version 2 still supports older versions of Oracle that are no longer available. +The community that had developed on GitHub, decided that a new internal architecture was needed, from that version 3 was born. + +We welcome new developers to join our community and contribute to the utPLSQL project. + +Primary features: + - Support for all basic scalar data-types + - Support for User Defined Object Types and Collections + - Support for native cursors both strong and weak + - Data-type aware testing + - [Annotations](docs/userguide/annotations.md) based test definitions + - Extensible [matchers](docs/userguide/expectations.md) + - Extensible reporting formats + - Extensible output providers + - Support for multi-reporting + - Code coverage reporting (with different formats) + - Runtime reporting of test execution progress + - Well-defined API + - Easy to call from current PL/SQL development tools + - More permissive License to allow vendors to integrate utPLSQL without violation of license + - Published upgrade/conversion path from version 2 ( TODO ) + +Requirements: + - Version of Oracle under [Extend Support](http://www.oracle.com/us/support/library/lifetime-support-technology-069183.pdf) (Currently 11.2 and above) __Version 2 to Version 3 Comparison__ @@ -81,84 +94,103 @@ sqlplus admin/admins_password@xe @@install_headless.sql For detailed instructions on other install options see the [Install Guide](docs/userguide/install.md) -# Annotations +# Example test package -Annotations provide a way to configure tests and suites in a declarative way similar to modern OOP languages. -The annotation list is based on moder testing framework such as jUnit 5, RSpec. +The below test package is a fully-functional Unit Test package for testing a function `betwnstr`. +Package specification is annotated with special comments ([annotations](docs/userguide/annotations.md)). +Annotations define that a package is a unit test suite, they also allow defining a description for the suite as well as the test itself. +Package body consists of procedures containing unit test code. To validate [an expectation](docs/userguide/expectations.md) in test, use `ut.expect( actual_data ).to_( ... )` syntax. -Annotations allow to configure test infrastructure in a declarative way without anything stored in tables or config files. The framework runner scans the schema for all the suitable annotated packages, automatically configures suites, forms hierarchy from then and executes them. -# Example of annotated package -``` -create or replace package test_pkg is +```sql +create or replace package test_between_string as + + -- %suite(Between string function) - -- %suite(Name of suite) - -- %suitepath(all.globaltests) + -- %test(Returns substring from start position to end position) + procedure normal_case; - -- %beforeall - procedure globalsetup; + -- %test(Returns substring when start position is zero) + procedure zero_start_position; - -- %afterall - procedure global_teardown; + -- %test(Returns string until end if end position is greater than string length) + procedure big_end_position; - /* Such comments are allowed */ + -- %test(Returns null for null input string value) + procedure null_string; +end; +/ - -- %test - -- %displayname(Name of test1) - procedure test1; +create or replace package body test_between_string as - -- %test(Name of test2) - -- %beforetest(setup_test1) - -- %aftertest(teardown_test1) - procedure test2; + procedure normal_case is + begin + ut.expect( betwnstr( '1234567', 2, 5 ) ).to_equal('2345') ); + end; - -- %test - -- %displayname(Name of test3) - -- %disable - procedure test3; - - -- %test(Name of test4) - -- %rollback(manual) - procedure test4; + procedure zero_start_position is + begin + ut.expect( betwnstr( '1234567', 0, 5 ) ).to_( equal('12345') ); + end; - procedure setup_test1; + procedure big_end_position is + begin + ut.expect( betwnstr( '1234567', 0, 500 ) ).to_( equal('1234567') ); + end; - procedure teardown_test1; + procedure null_string is + begin + ut.expect( betwnstr( null, 2, 5 ) ).to_( be_null ); + end; - -- %beforeeach - procedure setup; +end; +/ +``` - -- %aftereach - procedure teardown; -end test_pkg; +# Running tests + +To execute using IDE ()TOAD/SQLDeveloper/PLSQLDeveloper/other) just run the following. +```sql +begin + ut_runner.run(); +end; +/ ``` +Will run all the suites in the current schema and provide documentation report using dbms_output -#Annotations meaning +``` +Between string function + Returns substring from start position to end position + Returns substring when start position is zero + Returns string until end if end position is greater than string length + Returns null for null input string value + +Finished in .036027 seconds +4 tests, 0 failures +``` + +To execute your tests from command line, you will need a oracle sql client like SQLPlus or [SQLcl](http://www.oracle.com/technetwork/developer-tools/sqlcl/overview/index.html) +You may benefit from using the [ut_run.sql](client_source/sqlplus/ut_run.sql) to execute your tests if you want to achieve one of the following: +* see the progress of test execution for long-running tests +* have output to screen with one output format (text) and at the same time have output to file in other format (xunit) -| Annotation |Level| Describtion | -| --- | --- | --- | -| `%suite()` | Package | Marks package to be a suite of tests This way all testing packages might be found in a schema. Optional schema discription can by provided, similar to `%displayname` annotation. | -| `%suitepath()` | Package | Similar to java package. The annotation allows logical grouping of suites into hierarcies. | -| `%displayname()` | Package/procedure | Human-familiar describtion of the suite/test. Syntax is based on jUnit annotation: `%displayname(Name of the suite/test)` | -| `%test()` | Procedure | Denotes that a method is a test method. Optional test discription can by provided, similar to `%displayname` annotation. | -| `%beforeall` | Procedure | Denotes that the annotated procedure should be executed once before all elements of the current suite. | -| `%afterall` | Procedure | Denotes that the annotated procedure should be executed once after all elements of the current suite. | -| `%beforeeach` | Procedure | Denotes that the annotated procedure should be executed before each `%test` method in the current suite. | -| `%aftereach` | Procedure | Denotes that the annotated procedure should be executed after each `%test` method in the current suite. | -| `%beforetest()` | Procedure | Denotes that mentioned procedure should be executed before the annotated `%test` procedure. | -| `%aftertest()` | Procedure | Denotes that mentioned procedure should be executed after the annotated `%test` procedure. | -| `%rollback()` | Package/procedure | Configure transaction control behaviour (type). Supported values: `auto`(default) - rollback to savepoint (before the test/suite setup) is issued after each test/suite teardown; `manual` - rollback is never issued automatically. Property can be overridden for child element (test in suite) | -| `%disable` | Package/procedure | Used to disable a suite or a test | +Example: +``` +c:\my_work\>sqlplus /nolog @ut_run hr/hr@xe +``` +Will run all the suites in the current schema (hr) and provide documentation report into screen. +Invoking this script will show the progress after each test. __Primary Directories__ * .travis - contains files needed for travis-ci integration +* client_source - Sources to be used on the client-side. Developer workstation or CI platform to run the tests. +* development - Set of useful scripts and utilities for development and debugging of utPLSQL * docs - Markdown version of the documentation * examples - contains example unit tests. -* source - contains the code utPLSQL -* lib - 3rd party libraries that are required for source. +* source - contains the installation code for utPLSQL * tests - contains the tests written to test utPLSQL From 0aaace4050a57278591bb358d98748ed6e9ec336 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jacek=20G=C4=99bal?= Date: Wed, 18 Jan 2017 18:39:36 +0000 Subject: [PATCH 2/2] Updated doc link to point to Oracle 12.1 --- docs/userguide/expectations.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/userguide/expectations.md b/docs/userguide/expectations.md index 0663e035c..f17c2fefe 100644 --- a/docs/userguide/expectations.md +++ b/docs/userguide/expectations.md @@ -38,7 +38,7 @@ Usage: ut.expect( a_actual ).to_( match( a_pattern in varchar2, a_modifiers in varchar2 := null) ) ``` -Parameters `a_pattern` and `a_modifiers` represent a valid regexp pattern accepted by [Oracle regexp_like function](https://docs.oracle.com/cd/E11882_01/server.112/e41084/conditions007.htm#SQLRF00501) +Parameters `a_pattern` and `a_modifiers` represent a valid regexp pattern accepted by [Oracle regexp_like function](http://docs.oracle.com/database/121/SQLRF/conditions007.htm#SQLRF00501) ## equal