Feature/add reason to disable #1183

lwasylow · 2022-02-06T17:40:06Z

Adding option to put a reason for disabling tests.
CLOSES #610

…ture/add_reason_to_disable

github-actions · 2022-02-06T17:45:55Z

Unit Test Results

      1 files       2 suites 1m 38s ⏱️
1 029 tests 1 025 ✔️ 4 💤 0 ❌
1 032 runs 1 024 ✔️ 8 💤 0 ❌

Results for commit c762920.

♻️ This comment has been updated with latest results.

codecov · 2022-02-06T17:46:01Z

Codecov Report

Merging #1183 (c762920) into develop (ab1acfa) will decrease coverage by 0.06%.
The diff coverage is 100.00%.

@@             Coverage Diff             @@
##           develop    #1183      +/-   ##
===========================================
- Coverage    94.86%   94.80%   -0.07%     
===========================================
  Files          104      104              
  Lines         4853     4886      +33     
  Branches        21       23       +2     
===========================================
+ Hits          4604     4632      +28     
- Misses         241      245       +4     
- Partials         8        9       +1

Flag	Coverage Δ
11xe	`94.22% <100.00%> (+0.04%)`	⬆️
18xe	`94.44% <100.00%> (-0.09%)`	⬇️
21xe	`94.62% <100.00%> (-0.07%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
source/core/types/ut_run.tpb	`93.93% <ø> (ø)`
source/core/ut_suite_cache_manager.pkb	`97.01% <ø> (ø)`
source/core/ut_suite_manager.pkb	`95.81% <ø> (ø)`
source/core/ut_utils.pkb	`94.19% <ø> (ø)`
source/api/ut_suite_item_info.tpb	`100.00% <100.00%> (ø)`
source/core/types/ut_logical_suite.tpb	`100.00% <100.00%> (ø)`
source/core/types/ut_suite.tpb	`100.00% <100.00%> (ø)`
source/core/types/ut_suite_item.tpb	`100.00% <100.00%> (ø)`
source/core/types/ut_test.tpb	`100.00% <100.00%> (ø)`
source/core/ut_suite_builder.pkb	`99.69% <100.00%> (+<0.01%)`	⬆️
... and 7 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 08bb27b...c762920. Read the comment docs.

jgebal

We need to also update other reporters except ut_documentation_reporter
Particularly:

ut_junit_reporter - tag <skipper>reason</skipped>
ut_tfs_junit_reporter - tag <skipper>reason</skipped>

For the ut_realtime_reporter - It would be good to check with @PhilippSalvisberg
For the ut_teamcity_reporter - I'm not sure if the format supports a disabled/skipped description. The format description can be found here.
I didn't run utPLSQL on TeamCity for a long time and don't have local setup ready.
You would need to install and run TeamCity build locally (docker?) to actually test it.

Update
I've checked the documentation and the disabled (ignored) description is supported in TeamCity:

##teamcity[testIgnored name='testName' message='ignore comment']

source/core/ut_suite_builder.pkb

PhilippSalvisberg · 2022-02-06T21:28:52Z

We need to also update other reporters except ut_documentation_reporter Particularly:
(...)
For the ut_realtime_reporter - It would be good to check with @PhilippSalvisberg

I don't think this information is required for the realtime reporter. If you decide to add it, it will not have any effect. The SQLDev extension and the PL/SQL Developer plugin will ignore this additional information. If you think the behaviour in the IDE should change then I suggest to open dedicated issues for SQLDev and PL/SQL Developer.

jgebal · 2022-02-06T21:45:00Z

@lwasylow @simasch
Not sure what are your thoughts on having disabled reason visible in SQL/PLSQL Developer IDE.
I think it would be a good to have a complete feature and similar behaviour across all use-cases & IDEs.

lwasylow · 2022-02-07T09:34:19Z

@lwasylow @simasch Not sure what are your thoughts on having disabled reason visible in SQL/PLSQL Developer IDE. I think it would be a good to have a complete feature and similar behaviour across all use-cases & IDEs.

So personally I think if we want to have that feature enabled it makes sense to be consistent. I think it's useful piece of information.
However lets hold on with making that call to see if other reporters support the reason of disabling.
I don't think it should hold up release of that feature as we can stage it given a dependency between objects and time needed for @PhilippSalvisberg @simasch to give us specs ( and I mean their personal time :) ) but would ultimate aim for that to be included?

PhilippSalvisberg · 2022-02-07T10:11:57Z

@lwasylow @simasch Not sure what are your thoughts on having disabled reason visible in SQL/PLSQL Developer IDE. I think it would be a good to have a complete feature and similar behaviour across all use-cases & IDEs.

So personally I think if we want to have that feature enabled it makes sense to be consistent. I think it's useful piece of information. However lets hold on with making that call to see if other reporters support the reason of disabling. I don't think it should hold up release of that feature as we can stage it given a dependency between objects and time needed for @PhilippSalvisberg @simasch to give us specs ( and I mean their personal time :) ) but would ultimate aim for that to be included?

As mentioned before, you can include it if you want. The SQLDev extension and the PL/SQL Dev plugin will work without any change since they just pick the information from the XML events they need. I suggest to open dedicated issues for the IDEs. I think the commercial tools TOAD and DataGrip do not use the realtime reporter. However, if the reason for a disabled test should be displayed in the test runner they need to change something as well.

lwasylow · 2022-02-07T10:50:07Z

Please see snippet from junit reporter.

lwasylow · 2022-02-07T11:28:28Z

Here are results for ut_tfs_junit_reporter

as per documentation

Error message	/Testsuites/testsuite/testcase/failure.Attributes["message"].Value Or /Testsuites/testsuite/testcase/error.Attributes["message"].Value Or /Testsuites/testsuite/testcase/skipped.Attributes["message"].Value

simasch · 2022-02-07T12:25:43Z

If created a feature request for the PL/SQL Developer plugin utPLSQL/utPLSQL-PLSQL-Developer#25

lwasylow · 2022-02-07T13:07:18Z

Here results for ut_teamcity_reporter

sonarqubecloud · 2022-02-08T12:08:50Z

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

100.0% Coverage
0.7% Duplication

lwasylow added 3 commits February 6, 2022 15:49

Update code.

987d391

Updating testss

fddde60

Merge branch 'develop' of https://github.com/utPLSQL/utPLSQL into fea…

ca227c0

…ture/add_reason_to_disable

lwasylow requested review from jgebal and pesse and removed request for jgebal February 6, 2022 17:40

lwasylow linked an issue Feb 6, 2022 that may be closed by this pull request

Add support for reason in the --%disabled annotation #610

Closed

lwasylow requested a review from jgebal February 6, 2022 18:01

jgebal reviewed Feb 6, 2022

View reviewed changes

source/core/ut_suite_builder.pkb Show resolved Hide resolved

source/core/ut_suite_builder.pkb Outdated Show resolved Hide resolved

jgebal added this to the 3.1.12 milestone Feb 6, 2022

Resolving a code comments

79da316

lwasylow added 3 commits February 7, 2022 13:32

Updating reporters

ed156d7

Updates to junit xsd

2ea9a53

Update coverage

78cfb83

jgebal mentioned this pull request Feb 7, 2022

Add support for disabled reason with utPLSQL v3.1.12 utPLSQL/utPLSQL-SQLDeveloper#143

Closed

Update sonar

c762920

lwasylow requested a review from jgebal February 8, 2022 12:01

jgebal approved these changes Feb 8, 2022

View reviewed changes

lwasylow merged commit 62716ae into develop Feb 8, 2022

lwasylow deleted the feature/add_reason_to_disable branch February 8, 2022 12:55

PhilippSalvisberg mentioned this pull request Feb 25, 2022

Extend ut_realtime_reporter to include disabled reason #1192

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feature/add reason to disable #1183

Feature/add reason to disable #1183

lwasylow commented Feb 6, 2022 •

edited

Loading

github-actions bot commented Feb 6, 2022 •

edited

Loading

codecov bot commented Feb 6, 2022 •

edited

Loading

jgebal left a comment •

edited

Loading

PhilippSalvisberg commented Feb 6, 2022

jgebal commented Feb 6, 2022

lwasylow commented Feb 7, 2022 •

edited

Loading

PhilippSalvisberg commented Feb 7, 2022 •

edited

Loading

lwasylow commented Feb 7, 2022

lwasylow commented Feb 7, 2022

simasch commented Feb 7, 2022

lwasylow commented Feb 7, 2022

sonarqubecloud bot commented Feb 8, 2022

Feature/add reason to disable #1183

Feature/add reason to disable #1183

Conversation

lwasylow commented Feb 6, 2022 • edited Loading

github-actions bot commented Feb 6, 2022 • edited Loading

Unit Test Results

codecov bot commented Feb 6, 2022 • edited Loading

Codecov Report

jgebal left a comment • edited Loading

Choose a reason for hiding this comment

PhilippSalvisberg commented Feb 6, 2022

jgebal commented Feb 6, 2022

lwasylow commented Feb 7, 2022 • edited Loading

PhilippSalvisberg commented Feb 7, 2022 • edited Loading

lwasylow commented Feb 7, 2022

lwasylow commented Feb 7, 2022

simasch commented Feb 7, 2022

lwasylow commented Feb 7, 2022

sonarqubecloud bot commented Feb 8, 2022

lwasylow commented Feb 6, 2022 •

edited

Loading

github-actions bot commented Feb 6, 2022 •

edited

Loading

codecov bot commented Feb 6, 2022 •

edited

Loading

jgebal left a comment •

edited

Loading

lwasylow commented Feb 7, 2022 •

edited

Loading

PhilippSalvisberg commented Feb 7, 2022 •

edited

Loading