Can one of the admins verify this patch?

@shaded-enmity @pirat89 if you think that my opinion is wrong feel free to overrule my review.

@vojtechsokol +1 on what @vinzenz said, the much cleaner solution here is to inspect the result of _call and raise accordingly.

Note that besides the already mentioned points, this wouldn't work as expected once #481 is merged since the check for executable is done in the parent process. So if I _call'd with custom env={'PATH': ...}, the find_executable check could fail because it'd be looking in the wrong PATH.

Note _call actually still raises anythin that exec functions are raising and no error object is returned for this by _call

@shaded-enmity
I don't think inspecting the result of _call would help here. How would you distinguish the case when command fails with exit code 1 from the case when command does not exist? In both cases the result dictionary is the same, except for things that are worthless:

• signal: 0 in both cases
• pid: worthless
• exit_code: 1 in both cases
• stdout: worthless
• stderr: worthless (string with traceback for non-existitng command)

And regarding #481 - yes, that was expected, but not unsolvable, therefeore the WIP label.

@vinzenz

Yes, I'm raising exception not because the command failed, but because it's missing, as can be seen from message of raised exception. This is the intention of this PR - differentiate the case when command fails from case when command does not exist.

Regarding the case when path exists but is not executable - yes, I'm aware of that, buth that was first shot, therefeore the WIP label.

So what would you recommend then? Taking any action in stdlib.run function is too late, what remains is the _call function, but what exception should be transformed there?

Just looking into the original subprocess.call, it's a little tricky that the function raises different exceptions, depending on the Python you are using. In case of Python2, it raises just OSError with various messages. In case of Python3, every specific problem has own exception (PermissionError, FileNotFoundError). Honestly I like the Python3 way, but the other exceptions should be derived from the CalledProcessError, so actors that already work with that, would be able to catch it as it is without change.

Just looking into the original subprocess.call, it's a little tricky that the function raises different exceptions, depending on the Python you are using. In case of Python2, it raises just OSError with various messages. In case of Python3, every specific problem has own exception (PermissionError, FileNotFoundError)

Which are all derived from OSError https://docs.python.org/3/library/exceptions.html#exception-hierarchy

Honestly I like the Python3 way, but the other exceptions should be derived from the CalledProcessError, so actors that already work with that, would be able to catch it as it is without change.

I actually prefer if the exception is separate, as one is representing the side of the child process and the other one all things before that.

@vinzenz I just wonder if the parent/child separation is important enough to complicate the failure modes when check=True, I'd assume that most call sites of run don't really know/care about such distinction, but perhaps my assumption is wrong.

@vinzenz I just wonder if the parent/child separation is important enough to complicate the failure modes when check=True, I'd assume that most call sites of run don't really know/care about such distinction, but perhaps my assumption is wrong.

@shaded-enmity
I mean I would be fine if we would go with something like CommandExecutionError and our CalledProcessError being a derived class for it. But whatever we do, we will have to change already all the actors that are checking for OSError and may actually use it. e..g I use the distinction between OSError and CalledProcessError to decide what I am going to report to the user. e.g. Check that the thing is actually installed vs hmm maybe it's about your network, storage or whatever.

try:
    run()
except OSError as e:
    # report the user that the app is not there  or even if there's a workaround like trying to use a different program then go this route
except CalledProcessError as e:
    # Report stderr of the command (Or whatever)

If we would have a unified way one would end up with something like:

try:
    run()
except CalledProcessError as e:
    if e.exit_code is None: # Or whatever the moniker will be for this
      # hmm ok what was going on now? OSError will be stored inside?
    else:
      # Report stderr of the command (Or whatever)

Well, I am not sure why we actually need to do anything about this exception, you can use OSError and should be covered (from what I have seen but I am ok to be proven wrong)

For whose convenience is this? If you want to make a convenience you can make a tuple available that contains all the exceptions and people should catch that if they do not care about the specifics.

I don't want to make the life harder for anyone who does actually care about the difference.

from leapp.libraries.stdlib import run, LEAPP_RUN_ERRORS
...
try:
    run()
except LEAPP_RUN_ERRORS:
   pass

And for everybody else, you can keep CalledProcessError and OSError this way you don't break anything for anyone and allow people to catch what they need.

And about check=False, since this was supposed to be about the exit_code, OSError could still be thrown check=False wouldn't make the execution exception free, at least that's what I wouldn't except. If that is wanted there should be rather a 'noexcept' or whatever.

@vinzenz

If we would have a unified way one would end up with something like:

try:
    run()
except CalledProcessError as e:
    if e.exit_code is None: # Or whatever the moniker will be for this
      # hmm ok what was going on now? OSError will be stored inside?
    else:
      # Report stderr of the command (Or whatever)

It doesn't need to. It would end up with something like:

try:
    run()
except FileNotFoundError as e:
   # do what you want
except CalledProcessError as e:
   # do what you want

And in case they do not care about difference between those two errors, they still can use just

try:
    run()
except CalledProcessError as e:
    # to catch 'everywhing'

Well, I am not sure why we actually need to do anything about this exception, you can use OSError and should be covered (from what I have seen but I am ok to be proven wrong)

For whose convenience is this? If you want to make a convenience you can make a tuple available that contains all the exceptions and people should catch that if they do not care about the specifics.

I don't want to make the life harder for anyone who does actually care about the difference.

from leapp.libraries.stdlib import run, LEAPP_RUN_ERRORS
...
try:
    run()
except LEAPP_RUN_ERRORS:
   pass

And for everybody else, you can keep CalledProcessError and OSError this way you don't break anything for anyone and allow people to catch what they need.

That would be solution as well. Even when I would like to go more with Python3 exceptions. But that's just preference. I have not so big problem with that solution as well. But in such case we will have to refactor all caches in leapp-repository. In the unified way, we do not have to do any changes and it's quite easy to work with that as well, without problems.

@shaded-enmity just to keep you on the board :)

Well, I am not sure why we actually need to do anything about this exception, you can use OSError and should be covered (from what I have seen but I am ok to be proven wrong)
For whose convenience is this? If you want to make a convenience you can make a tuple available that contains all the exceptions and people should catch that if they do not care about the specifics.
I don't want to make the life harder for anyone who does actually care about the difference.

from leapp.libraries.stdlib import run, LEAPP_RUN_ERRORS
...
try:
    run()
except LEAPP_RUN_ERRORS:
   pass

And for everybody else, you can keep CalledProcessError and OSError this way you don't break anything for anyone and allow people to catch what they need.

That would be solution as well. Even when I would like to go more with Python3 exceptions. But that's just preference. I have not so big problem with that solution as well. But in such case we will have to refactor all caches in leapp-repository.

The point is that you don't have to refactor anything at this point, it's a can be updated by someone touching the actor. If you start introducing new Exceptions however that's a different story. Then you have to refactor everything also it's not so easy to make our exception + a builtin exception. I don't really see why you feel like we have to do anything at all. The only thing you can introduce is that tuple that has OSError and CalledProcessError and it should catch all the possible exceptions from run() - Theres no difference right now. Already everyone should be catching OSError at this point, if they don't then their code didn't handle the exceptions properly already.

In the unified way, we do not have to do any changes and it's quite easy to work with that as well, without problems.

In the unified way you have to refactor every code, that actually handles them separately already.
While if you do just introduce a tuple to catch everything, only new code should use it.

@vinzenz
As stated in the PR description - catching OSError and CalledProcessError is not enough, because this would lead to a traceback with KeyError.

Now the code should properly handle the cases when the path exists but is not executable, and when environment variable with PATH is used.

@vinzenz man, I am not getting your answer. I see that in completely opossite way, when we will have to change something and when not. What are you writting, looks to me like you missed something in my answer or I am missing something in your answer. If you have:

class CalledProcessError(LeappError):
   ...

class FileNotFoundError(CalledProcessError):
   ...

class ...

Why I need to touch every call that catches CalledProcessError? The behaviour for them is same + catching errors when command doesn't exist. In case they care about the reason more, they can add catch of additional error. This means in >90% of cases it doesn't need any change from us. In your case, it's opposite as you need to change import and catch in >90%. Taking the point, that probably everyone who used catch of CalledProcessError expect that the only thing they need to do and don't know that there are other cases. (other cases generate nowadays raw, unhandled tracebacks in actors instead of clear error messages for users).

Additionally, generated OSErrors, etc. in this solution will point more to a bug in leapp stdlib instead of problem in command itself. That would be beneficial for us as well.

I think we are desynced here. Additional discussion through comments would be too slow, so I would prefer mtg to discuss it in case of need.

EDIT: fixed copy&paste error in the pseudocode
EDIT2: Or keep it on cabal mtg ;-)

The actual cause of the Traceback is handled now by #533