Hi @pesse
So we would have to do some validation to ensure that tag is not starting with '-' as such in annotation cache and treat the ^-.% as reserved.
We would basically execute all tests ( tagged or not ) that do not have a tag -tag or have no tag at all.
Is that correct ?

@lwasylow
I think we should only allow single_word tag names.
It is to be considered if they should be case sensitive or not.
to be honest I'm not sure if they are considered case-sensitive today.

Also, it looks like we silently remove/ignore tags that are not single_words with this line of code:
a_list := ut_utils.convert_collection( ut_utils.filter_list(l_tag_list,ut_utils.gc_word_no_space) );

We should probably put warnings on tag names that are not valid (before we filter them out).
We should also put warning and ignore tag names that start with a "-" symbol.

Also, it looks like we silently remove/ignore tags that are not single_words with this line of code:
a_list := ut_utils.convert_collection( ut_utils.filter_list(l_tag_list,ut_utils.gc_word_no_space) );

This is due to:

Tags should adhere to following rules:

* tags are case sensitive
* tags cannot be an empty string
* tags cannot contain spaces e.g. to create a multi-word tag please use underscores,dashes, dots etc. e.g. test_of_batch
* tags with empty spaces will be ignored during execution
* tags can contain special characters

(given from: http://utplsql.org/utPLSQL/v3.1.7/userguide/annotations.html)

Yes, will fix that and add a warning about it.

I support @pesse 's approach of adding "minus" as a prefix to the tags for the tests that must be excluded:

ut.run(a_tags=>'wip,-long_running');

We would basically execute all tests ( tagged or not ) that do not have a tag -tag or have no tag at all.
Is that correct ?

Yes, as to me this is correct. Consider a use case scenario: test case is marked as

--%tag(long_running)

and other test cases have other tags or have no tags at all.
When I call:

ut.run(a_tags=>'-long_running');

then I expect that will be executed ALL test cases EXCEPT those marked with "long_running" tag.
When I call:

ut.run(a_tags=>'wip,-long_running');

then I expect that will be executed ALL test cases EXCEPT those marked with "long_running" tag PLUS those that marked with "wip".

In a conflict situation, when we have intersecting sets:

--%tag(wip)
procedure test_wip;

--%tag(wip,long_running)
procedure test_wip_long;

--%tag(long_running)
procedure test_long;

and when we call:

ut.run(a_tags=>'wip,-long_running');

or

ut.run(a_tags=>'-long_running,wip');

then I expect that will be ALL test cases EXCEPT those marked with "long_running" tag PLUS those that marked with "wip" EXCEPT those marked with "long_running" again. I.e. in example above will be run only

--%tag(wip)
procedure test_wip;

But now this MY expectation goes against the "OR" rule of annotations (http://utplsql.org/utPLSQL/v3.1.7/userguide/annotations.html):

Framework applies OR logic when resolving a tags so any tests / suites that match at least 
one tag will be included in the test run.

That's why I'd considered a change to the "OR" rule for annotations and made it this way:

Framework applies OR logic when resolving "positive" tags (without "minus" prefix) and 
at the end applies AND NOT logic when resolving "negative" tags (with "minus" prefix). 
So processing for the scope for the test run is in the following order:
1.  ("OR" rule, legacy): any tests / suites that match at least one "positive" tag will be 
included in the test run.
2. ("AND NOT" rule, new feature): any tests / suites that match at least one "negative" tag 
will be excluded from the test run. 
3. (special case caused by new feature): all "positive" vs. "negative" conflicts will be 
resolved to satisfy "negative" tags because the rule "AND NOT" is applied at the end 
of the scope formation.

What would you say?

I would say:
Exclusion is stronger than inclusion and so in your scenario:

--%tag(wip)
--%test
procedure test_wip;

--%tag(wip,long_running)
--%test
procedure test_wip_long;

--%tag(long_running)
--%test
procedure test_long;

and when we call ut.run(a_tags=>'wip,-long_running'); or ut.run(a_tags=>'-long_running,wip');, only the test_wip test gets executed.

This is because of the exclusion logic.

So we say:

• include only wip items tagged as wip and exclude all long_running

The or logic is applied to inclusion/exclusion list only.
When applying inclusions/exclusions we would use set-based logic that is similar to this operation

select item from items where tag = 'wip'
minus
select item from items where tag = 'long_running'

When you combine filtering by suitepath and filtering by tag and operator is used.
So when you run a suite, only suites that are on the specified suitepath AND contain any of specified tags will be executed.

@lwasylow - I'm already making changes for tags

@jgebal, I totally agree with you saying:

I would say:
Exclusion is stronger than inclusion and so in your scenario:
...

And thank you very much for making changes for tags already! Tags are very important for automation tools usages.

@alexeyhimself
Changes are now merged into 3.1.9 develop
Will be leased soon.

@jgebal, thank you very much!

Also, cli already supports it. You can just go

utplsql run user/pw@connection --tags=-long_running

with 3.1.8 cli against 3.1.9 core

@pesse, yes, I've seen that in utPLSQL-cli repo! And I really appreciate that! Thank you!
We will try the feature as soon as utPLSQL v3.1.9 appear!