This implementation looks nice and clean. Just some minor inline comments:

The reason I like this patch is that I've by now repeatedly told other PR senders that "No, you're not allowed to have True as a default for your BoolParameter". But with this patch we could allow it from now on.

This is a reasonably big change (not in code size), but in what we communicate are good practices to the luigi users. I remember @erikbern saying some time that BoolParameters should always be false by default. But I don't know if he meant that as a "that's good design in general" or "due to current luigi limitations that's the only implementation that works out".

Understood. I agree that existing community usages may have a part to play in whether this PR should be accepted or not.

As an anecdotal data point, at my employer, we use Luigi and I have seen our developers confused on multiple occasions by not really being able to set default=True on BoolParameter. (The confusion stemming from the BoolParameter behaving differently from other Parameter subclasses with respect to setting its value.)

And this PR is an attempt to improve the API to remove that source of confusion.

I agree with @tdyas on this, as someone who used Luigi heavily while working with @erikbern . It doesn't break the existing semantics, and it does exactly what I had assumed it would have done when I first started using Luigi: adds a --no-xyz argument, which is a de-facto standard for GNU tools and elsewhere.

The only reason default=True was a bad practice is because it lacked this implementation, and we had to do dumb hacks like create argument variables named no_this and disable_that.

Thank you both @tdyas and @a1k0n for sharing your design thoughts, they sound sane to me. I'll just give @erikbern until tomorrow to chime in. Otherwise I'll merge this tomorrow once you fix the build.

i'm fine with this

Adding the --no-xyz arguments seems to confuse argparse's parse_known_args method. I'll have to dig into why argparse is confused.

runtests.py: error: ambiguous option: --n could match --no-scheduler-record-task-history, --no-lock, --no-worker-no-install-shutdown-handler, --no-no-lock, --no-RangeBase-reverse, --no-worker-count-uniques, --no-scheduler-prune-on-get-work, --no-RangeDaily-reverse, --no-local-scheduler, --no-worker-retry-external-tasks, --no-RangeHourlyBase-reverse, --no-take-lock, --no-RangeDailyBase-reverse, --no-scheduler-prune-done-tasks, --no-RangeHourly-reverse, --no-parallel-scheduling, --no-TestNotificationsTask-raise-in-complete, --no-worker-keep-alive, --no-assistant

i'm guessing no argument can be a prefix of another

So I dug in further, and it seems like a bug in argparse. The error occurs during the call to parse_known_args in CmdlineParser.__init__. That method is trying to determine the root_task to use. argparse should really not throw an error about an ambiguous option when invoked via parse_known_args. It should just mark the option in question as unknown.

I can get the failing test to pass by renaming the n option in SomeTask in cmdline_test.py to something else (e.g., z).

This still brings up an issue about whether this will break Luigi for anybody who has a task with a "n" parameter.

And this issue is known to the Python developers: https://bugs.python.org/issue19814

And apparently only warrants a warning in the documentation.

Please let me know if you need any help. Otherwise I'm patiently waiting for something that builds. :)

As it turns out, the test I added wasn't actually testing whether a BoolParameter with default=True could be set to False by option. This uncovered some places where checks for truthy values had to actually check for None.

I also merged all of the other commits (other than the commit refactoring the add_argument call) into the commit adding the inverted argument support.

There was also a test failure locally for me with test/file_test.py which seems unrelated to my change since the tests in that file don't deal with parameters. It could be an issue with my local setup so will wait for the Travis build to see if that is the case.

CI tests passed.

I'm still a bit worried about we suddenly dropping support for parameters with names like n and I'm not fond of you renaming a local variable to an all caps name (n --> N). Would it not just be much easier if you only had the --no-boolparam option only if default=True? Alternatively you could another option add_negative_cli_flag with a value False as default.

Yea I understand this is entirely the fault of some stupid bug in the argparse module (I've also felt the pain). But I'm afraid people are going to start to say their stuff broke and this would need to be reverted in haste if we break compatibility.

apparently python 3.5 added a flag to argparse allow_abbrev=False that turns off the prefix matching. too bad it wasn't earlier

the correct way would probably be to first change luigi's behavior so that prefix flags throw a deprecation warning for a few months and then an exception

btw reminds me – maybe it's time to publish a new version of luigi to pypi?

So the solution for now it seems is to just add a --no-FOO option for BoolParameter's with default=True?

Yea. I would be more comfortable that way actually @tdyas

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. If closed, you may revisit when your time allows and reopen! Thank you for your contributions.

Resolved by #2427