From 2a26454c6b26c45c9b4b1f1ef1ddd8b51f0e39be Mon Sep 17 00:00:00 2001 From: Ian Jackson Date: Tue, 5 Jan 2021 14:34:10 +0000 Subject: [PATCH 01/34] Propose Command improvements --- text/0000-command-ergonomics.md | 525 ++++++++++++++++++++++++++++++++ 1 file changed, 525 insertions(+) create mode 100644 text/0000-command-ergonomics.md diff --git a/text/0000-command-ergonomics.md b/text/0000-command-ergonomics.md new file mode 100644 index 00000000000..85ebb91b571 --- /dev/null +++ b/text/0000-command-ergonomics.md @@ -0,0 +1,525 @@ +- Feature Name: (fill me in with a unique ident, `my_awesome_feature`) +- Start Date: (fill me in with today's date, YYYY-MM-DD) +- RFC PR: [rust-lang/rfcs#0000](https://github.com/rust-lang/rfcs/pull/0000) +- Rust Issue: [rust-lang/rust#0000](https://github.com/rust-lang/rust/issues/0000) + +# Summary +[summary]: #summary + +Improve the ergonomics of `std::process::Command`, +and better protect programmers from error handling mistakes. + +This RFC is principally an attempt to +gain consensus on a fleshed out version of +the MR [#89004](https://github.com/rust-lang/rust/pull/89004) +which proposed to add some convenience APIs to `std::process::Command`. + +# Motivation +[motivation]: #motivation + +The current API for `std::process::Command` +makes it unnecessarily difficult to perform common tasks, +such as "simply running a program", +and running a program and collecting its output. + +The APIs that are currently provided invite mistakes +(for example, failing check the exit status in `Output`, +and deadlock errors). +Writing code that is fully correct +and produces good error messages is cumbersome, +and sometimes subtle. + +Running subprocesses is inherently complex, and provides many opportunites for errors to occur - so there are some questions about how to best represent this complexity in a convenient API. + +Existing work in this area has been fragmented into a number of +MRs and issues, making it hard to see the wood for the trees, +and has become bogged down due to lack of decision +on the overall approach. Some references: + + * [#84908](https://github.com/rust-lang/rust/issues/84908) + Stabilisation tracking issue for `ExitStatusError` + * [#81452](https://github.com/rust-lang/rust/pull/81452) + MR, closed as blocked: Add `#[must_use]` to `process::Command`, `process::Child` and `process::ExitStatus` + * [#89004](https://github.com/rust-lang/rust/pull/89004) + MR, closed: Add convenience API to `std::process::Command` + * [#88306](https://github.com/rust-lang/rust/pull/88306) + MR, closed: `ErrorKind::ProcessFailed` and `impl From` + * [#93565](https://github.com/rust-lang/rust/pull/93565) + MR, draft: `impl Try for ExitStatus` + * [#73126](https://github.com/rust-lang/rust/issues/73126) + issue: `std::process::Command` `output()` method error handling hazards + * [#73131](https://github.com/rust-lang/rust/issues/73131) + Overall tracking Issue for `std::process` error handling + +## Currently-accepted wrong programs + +The following incorrect program fragments are all accepted today and run to completion without returning any error: + +``` + Command::new("touch") + .args(&["/dev/enoent/touch-1"]); + // ^ programmer surely wanted to actually run the command + + Command::new("touch") + .args(&["/dev/enoent/touch-2"]) + .spawn()?; + // ^ accidentally failed to wait, if programmer wanted to make a daemon + // or zombie or something they should have to write let _ =. + + Command::new("touch") + .args(&["/dev/enoent/touch-3"]) + .spawn()? + .wait()?; + // ^ accidentally failed to check exit status +``` + +Corrected versions tend to have lots of boilerplate code, +especially if good error messages are wanted. + +# Guide-level explanation +[guide-level-explanation]: #guide-level-explanation + +We will introduce new APIs on `Command`, +for running the command and collecting its output: + +``` +impl Command { + fn run(&mut self) -> Result<(), SubprocessError>; + fn get_output_bytes(&mut self) -> Result, SubprocessError>; + fn get_output(&mut self) -> Result; + fn get_output_line(&mut self) -> Result; + fn get_output_read(&mut self) -> impl std::io::Read; +} +struct SubprocessError { ... } +impl From for io::Error { ... } +``` + +The `.output()` function and `std::process::Output` +will be deprecated. + +No significant changes are made to the `Command` construction APIs, +but it may become necessary to call `.stderr()` explicitly +to use the new methods (see Unresolved Questions). + +## Use cases + +We aim to serve well each of the following people: + + * Alice writes a CLI utility to orchestrate processes and wants top-notch error reporting from subprocesses. + + * Bob migrates ad-hoc automation scripts from bash to Rust. + + * Carol writes a generic application and just wants all errors to be maximally useful by default. + + * Dionysus wants to run `diff`, which exits `0` for "no difference", + `1` for "difference found" and + another value for failure. + +(Partly cribbed from one of +@matklad's [comments](https://github.com/rust-lang/rust/pull/89004#issuecomment-923803209) in #89004) + +# Reference-level explanation +[reference-level-explanation]: #reference-level-explanation + +## New methods on `Command`: + + * `fn run(&mut self) -> Result<(), SubprocessError>`: + + Runs the command. + Equivalent to `.spawn()` followed by `.status()`, + but with better error handling. + + * `fn get_output_bytes(&mut self) -> Result, SubprocessError>`: + + Runs the command and collects its stdout. + After the child indicates EOF on its stdout, + we will wait for it to finish and check the exit status. + + * `fn get_output(&mut self) -> Result, SubprocessError>`: + + Runs the command and collects its stdout. + Decodes the stdout as UTF-8, and fails if that's not possible. + Does not trim any trailing line ending. + + * `fn get_output_line(&mut self) -> Result, SubprocessError>`: + + Runs the command and collects its stdout. + Decodes the stdout as UTF-8, and fails if that's not possible. + Fails unless the output is a single line (with or without line ending). + Trims the line ending (if any). + + * `fn get_output_read(&mut self) -> std::process::ChildOutputStream` + (where `struct ChildOutputStream` implements `io::Read` + and is `Send + Sync + 'static`). + + Starts the command, allowing the caller to + read the stdout in a streaming way. + Neither EOF nor an error will be reported by `ChildOutputStream` + until the child has exited, *and* the stdout pipe reports EOF. + (This includes errors due to nonempty stderr, + if stderr was set to `piped`.) + +Most callers should use these methods, +rather than `.spawn()` or `.status()`. +These new methods should be be +recommended by docs for other, lower-level functions +which people were previously required to use. + +## Deprecations + + * Deprecate `std::process::Command::output()`. + This API cannot be fixed; + see [#73126](https://github.com/rust-lang/rust/issues/73126). + + * Apply `#[must_use]` to `Command` and `Child`. + + * Apply `#[must_use]` to `ExitStatus`. + (May require fixing quite a few of the examples.) + +## stderr handling + +If `stderr(Stdio::piped())`, +these new functions all collect the child's stderr. +Then, +if the stderr output is nonempty, this is considered an error, +and reported in the `SubprocessError`. + +These functions *do not* wait for EOF on stderr. +Rather, they wait for child process termination and expect that +any relevant error messages have been printed by that point. +Any further stderr output +(for example, from forked but unawaited children of the command) +might not be reported via the Rust std API. +Such output might be ignored, +or might accumulate in a temporary disk file not deleted +until the last escaped handle onto the file has gone. +Ideally, any escaped writer(s) would experience broken pipe errors. + +This non-waiting behaviour on stderr could be important +when Rust programs invoke tools like `ssh`, +which sometimes hang onto their stderr well after the +intended remote command has completed. + +The implementation may involve a temporary file, +or an in-memory buffer, +or both. + +## New `struct SubprocessError` + +This new struct is used as the error type for the new methods. + +It can represents zero or more of the various +distinct problems that can occur while running a process. +A `SubprocessError` returned by a `std` function will always +represent at least one problem (unless otherwise stated), +but it may represent several + +For example a process which exited nonzero +probably printed to stderr; +with `piped` we capture that, and represent both the stderr +and the exit status as problems within the `SubprocessError`. + +``` +impl SubprocessError { + /// The program, if we know it. + fn program(&self) -> Option<&OsStr>; + + /// The arguments, if we know them. + fn args(&self) -> Option>; + + /// If the stdout was captured in memory, the stdout data. + fn stdout_bytes(&self) -> Option<&[u8]>; + + /// If the process exited and we collected its status, the exit status. + fn status(&self) -> Option; + + /// If trouble included nonempty stderr, the captured stderr + fn stderr_bytes(&self) -> Option<&[u8]>; + + /// If trouble included failure to spawn, the spawn error. + fn spawn_error() -> Option<&io::Error>; + + /// If trouble included failure to talk to the child, the IO error. + /// + /// This might include problems which might be caused by child + /// misbehaviour. + fn communication_error() -> Option<&io::Error>; + + /// If trouble included failed UTF-8 conversion. + fn utf8_error(&self) -> Option<&std::str::FromUtf8Error>; +} +``` + +The `Display` implementation will print everything above +including the command's arguments. +The arguments will be escaped or quoted in some way that renders +a resulting error message unambiguous. + +### `impl From for io::Error` + +`SubprocessError` must be convertible to `io::Error` +so that we can use it in `ChildOutputStream`'s +`Read` implementation. +This may also be convenient elsewhere. + +The `io::ErrorKind` for a `SubprocessError` will be: + + * The `io::ErrorKind` from the spawn error, if any. + + * Otherwise, a new kind `io::ErrorKind::ProcessFailed`, + which means that the subprocess itself failed. + +### Further necessary APIs for `SubprocessError` + +We also provide ways for this new error to be constructed, +which will be needed by other lower level libraries besides std, +notably async frameworks: + +``` +impl SubprocessError { + /// Makes a "blank" error which doesn't contain any useful information + /// + /// `has_problem()` will return `false` until one of the setters + /// is used to store an actual problem. + fn new_empty() -> Self { } + + // If we keep ExitStatusError + fn from_exit_status_error(status: ExitStatusError) -> Self { } + + fn set_program(&mut self, impl Into); + fn set_args(&mut self, impl IntoIterator>); + fn set_stdout_bytes(output: Option>>); + + fn set_status(&mut self, status: ExitStatus); + fn set_stderr_bytes(&mut self, stderr: impl Into>); + fn set_spawn_error(&mut self, error: Option); + fn set_communication_error(&mut self, error: Option); + fn set_utf8_error(&mut self, error: Option); + + /// Find out if this error contains any actual error information + /// + /// Returns `false` for a fresh blank error, + /// or `true` for one which has any of the error fields set. + /// (Currently equivalent to checking all of `status()`, + /// `stderr_bytes()`, `spawn_error()` and `utf8_error()`.) + // + // We must provide this because it's needed for handling programs + // with unusual exit status conventions (eg `diff(1)`) + // and a caller can't reimplement it without making assumptons + // about `SubprocessError`'s contents. + fn has_problem(&self) -> bool; +} +impl Default for SubprocessError { ... } +impl Clone for SubprocessError { ... } // contained io:Errors are in Arcs +``` + +# Drawbacks +[drawbacks]: #drawbacks + +This is nontrivial new API surface. + +Much of the new API surface is in `SubprocessError`. +If we didn't want to try to make it easy for Rust programmers +to run subprocesses and produce good error messages, +we could omit this error type +(perhaps using something like `ExitStatusError`). + +Perhaps we don't need all the `get_output` variants, +and could require Bob to write out the boilerplate +or provide his own helper function. + +Perhaps we don't need `get_output_read`. +However, +avoiding deadlocks when reading subprocess output, +and also doing error checks properly, +is rather subtle. + +# Rationale and alternatives +[rationale-and-alternatives]: #rationale-and-alternatives + +Alternatives and prior proposals include: + + * `ExitStatusError` (tracking issue [#84908](https://github.com/rust-lang/rust/issues/84908)) + + Currently, exists but unstable. + Ergonomics of using this to produce good error messages are rather poor, + because the `ExitStatusError` is just the exit status. + + * `impl Try for ExitStatus` + [#93565](https://github.com/rust-lang/rust/pull/93565) + + libs-api team are + "[hesitant](https://github.com/rust-lang/rust/pull/93565#issuecomment-1367557592) to add more `Try` implementations". + Like `ExitStatusError`, it is difficult to see how this could produce good error messagesx + without a lot of explicit code at call sites. + + * Previous attempt at `Command::run()` and `Command::read_stdout()` + [#89004](https://github.com/rust-lang/rust/pull/89004). + + Seemed to be going in the right direction + but got bogged down due to lack of consensus on overall direction + and some bikeshed issues. + +# Prior art +[prior-art]: #prior-art + +Many other languages have richer or more convenient APIs +for process invocation and output handling. + + * Perl's backquote operator has the command inherit the script's stderr. If you want to do something else you need to do a lot of hand-coding. It reports errors in a funky and not particularly convenient way (but, frankly, that is typical for Perl). It doesn't trim a final newline but Perl has a chomp operator that does that very conveniently. + + * Tcl's `exec` captures stderr by default (there are facilities for redirecting or inheriting it), calling any stderr an error (throwing a Tcl exception). It always calls nonzero exit status an error (and there is no easy way to get the stdout separately from the error in that situation). It unconditionally chomps a final newline. + + * Python3's `subprocess.run` fails to call nonzero exit status an exception. If you don't write explicit error handling code (highly unusual in Python) you have an unchecked exit status bug. + +# Unresolved questions +[unresolved-questions]: #unresolved-questions + +## Printing command arguments in `impl Display for SubprocessError` + +Perhaps printing the command arguments is overly verbose, +and we should print onliy the command name. + +## `get_output` vs `read_output` naming. + +We have `fs::read_to_string`. + +Possible names (taking `get_output_bytes` as the example): + + * `get_output_bytes` (proposed in this RFC) + * `output_bytes` but `output` is alread taken for bad `Output`. + * `run_get_output_bytes` + * `run_output_bytes` + * `read_output_bytes` + +## stderr handling default + +What should happen if the Rust programmer didn't call `.stderr()` ? + +Options are: + + 1. Treat it as `inherit`. + This is probably fine for a command line application, + but may be too lax for many other contexts. + + 2. Treat it as `piped`: call any stderr output an error. + This is a fairly conservative choice, + but it can lead to unexpected runtime errors, + if a called program later starts printing warning messages. + + 3. Call this a programming mistake, and panic. + + 4. Somehow make this a compile error. + This would involve a new `CommandForWhichWeHaveSpecifiedStderr` + type (typestate pattern), or providing the stderr handling as a mandatory argument + to all the new methods. + These seem unpalatably unergonomic. + +Here we propose option 1: treat as `inherit`. + +# Future possibilities +[future-possibilities]: #future-possibilities + +## Changes to `ExitStatusError` + +Possibilities include: + + * Abolish `ExitStatusError` + * Stabilise `ExitStatusError` as-is + * `impl From for SubprocessError` + * `impl From for io::Error` + +Error messages from `ExitStatusError` are rather poor, +and it is likely that `SubprocessError` will +subsume most of its use cases. + +## Async ecosystem could mirror these APIs + + * An async versions of `run()` seems like it would be convenient. + * Async versions of the output-capturing runners too. + * Async frameworks ought to (be able to) use `SubprocessError`. + +Maybe `SubprocessError` would have to be able to contain a nested +`Arc`. +That doesn't need to happen now. +But it is one reason why `.has_problem()` needs to exist. + +## More flexible and less synchronous output handling + +We could provide a more concurrent API, +which allows a Rust program to experience the outcomes of +running a subprocess +(stdout output, stderr output, exit status) +as a series of events or callbacks, +and interleave waiting for the process with writing to its stdin. + +Options might include: + + * Allow the Rust programmer to supply an implementation of `Write` for process stdout for handling process stdout and stderr, via a new constructor `std::process::Stdio::from_write`. + + * Provide and stabilise something like cargo's internal function + [`read2`](https://github.com/rust-lang/cargo/blob/58a961314437258065e23cb6316dfc121d96fb71/crates/cargo-util/src/read2.rs) + + * Expect users who want this to use pipes by hand (perhaps with threads), or async. + +## More convenient way to run `diff` + +With the proposed API, +completely correctly running `diff(1)` would look a bit like this: + +``` + let result = Command::new("diff") + .args(["before","after"]) + .run(); + let status = match result { + Ok(()) => 0, + Err(err) => { + let status = err.status(); + err.set_status(ExitStatusExt::from_raw(0)); + if err.has_problem() { + return Err(err); + } + status.code() + } + }; +``` + +This is doable but cumbersome. +A naive Dionysus is likely to write: + +``` + let status = match result { + Ok(()) => 0, + Err(err) => { + if ! err.status().success() { + err.status().code() + } else { + return Err(err); + } + } + }; +``` + +As it happens, this is correct in the sense that it won't malfunction, +since actually `run()`, without piped stderr, +cannot produce a `SubprocessError` +containing a nonzero exit status *and* any other problem. +But in a more complex situation it might be wrong. + +Perhaps: +``` +impl SubprocessError { + /// Returns `Ok` if the only reason for the failure was a nonzero exit status. Otherwise returns `self`. + //// + /// Use this if you want to to tolerate some exit statuses, but still fail if there were other problems. + pub fn just_status(self) -> Result; +} +``` + +Then Dionysus can write: +``` + let status = match result { + Ok(()) => 0, + Err(err) => err.just_status()?.code(), + }; +``` From 9a4fc3415a314e06bc280a00e795c2d1d3ab1068 Mon Sep 17 00:00:00 2001 From: Ian Jackson Date: Wed, 4 Jan 2023 10:19:38 +0000 Subject: [PATCH 02/34] Apply rust tag to example Co-authored-by: Lucius Hu <1222865+lebensterben@users.noreply.github.com> --- text/0000-command-ergonomics.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-command-ergonomics.md b/text/0000-command-ergonomics.md index 85ebb91b571..cc442f733de 100644 --- a/text/0000-command-ergonomics.md +++ b/text/0000-command-ergonomics.md @@ -55,7 +55,7 @@ on the overall approach. Some references: The following incorrect program fragments are all accepted today and run to completion without returning any error: -``` +```rust Command::new("touch") .args(&["/dev/enoent/touch-1"]); // ^ programmer surely wanted to actually run the command From 89705df57a0fc77b9efa076649f31e8d8b2ed3f4 Mon Sep 17 00:00:00 2001 From: Ian Jackson Date: Wed, 4 Jan 2023 10:41:47 +0000 Subject: [PATCH 03/34] Discuss some alternatives around SubprocessError --- text/0000-command-ergonomics.md | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/text/0000-command-ergonomics.md b/text/0000-command-ergonomics.md index cc442f733de..0c0a9d22a9b 100644 --- a/text/0000-command-ergonomics.md +++ b/text/0000-command-ergonomics.md @@ -360,6 +360,23 @@ Alternatives and prior proposals include: but got bogged down due to lack of consensus on overall direction and some bikeshed issues. + * The `SubprocessError` type could be a + transparent non-exhaustive struct. + This might be reasonable, + since it's really just a bag of information + with getters and setters for every field. + But transparent structs are unfashionable in modern Rust. + + * Instead of a single `SubprocessError` type used everywhere, + there could be a different error type for the different calls. + For example, `run()` could have a different error type to + `get_output()`, + since `run` doesn't need to represent UTF-8 conversion errors. + This would not be very in keeping with the rest of `std::process`, + which tends to unified types with variation selected at runtime. + There would still have to be *a* type as complex as `SubprocessError`, + since that's what `get_output_read` needs. + # Prior art [prior-art]: #prior-art From 38b9c2a963683a3827cbeda336b54120ce59ec77 Mon Sep 17 00:00:00 2001 From: Ian Jackson Date: Wed, 4 Jan 2023 10:44:53 +0000 Subject: [PATCH 04/34] Add rust tags to many other code blocks --- text/0000-command-ergonomics.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/text/0000-command-ergonomics.md b/text/0000-command-ergonomics.md index 0c0a9d22a9b..f13154c42eb 100644 --- a/text/0000-command-ergonomics.md +++ b/text/0000-command-ergonomics.md @@ -82,7 +82,7 @@ especially if good error messages are wanted. We will introduce new APIs on `Command`, for running the command and collecting its output: -``` +```rust impl Command { fn run(&mut self) -> Result<(), SubprocessError>; fn get_output_bytes(&mut self) -> Result, SubprocessError>; @@ -219,7 +219,7 @@ probably printed to stderr; with `piped` we capture that, and represent both the stderr and the exit status as problems within the `SubprocessError`. -``` +```rust impl SubprocessError { /// The program, if we know it. fn program(&self) -> Option<&OsStr>; @@ -275,7 +275,7 @@ We also provide ways for this new error to be constructed, which will be needed by other lower level libraries besides std, notably async frameworks: -``` +```rust impl SubprocessError { /// Makes a "blank" error which doesn't contain any useful information /// @@ -484,7 +484,7 @@ Options might include: With the proposed API, completely correctly running `diff(1)` would look a bit like this: -``` +```rust let result = Command::new("diff") .args(["before","after"]) .run(); @@ -504,7 +504,7 @@ completely correctly running `diff(1)` would look a bit like this: This is doable but cumbersome. A naive Dionysus is likely to write: -``` +```rust let status = match result { Ok(()) => 0, Err(err) => { @@ -524,7 +524,7 @@ containing a nonzero exit status *and* any other problem. But in a more complex situation it might be wrong. Perhaps: -``` +```rust impl SubprocessError { /// Returns `Ok` if the only reason for the failure was a nonzero exit status. Otherwise returns `self`. //// @@ -534,7 +534,7 @@ impl SubprocessError { ``` Then Dionysus can write: -``` +```rust let status = match result { Ok(()) => 0, Err(err) => err.just_status()?.code(), From 2ff775e92cb66e3560c3776f949a64d4ea7793d0 Mon Sep 17 00:00:00 2001 From: Ian Jackson Date: Fri, 13 Jan 2023 13:02:47 +0000 Subject: [PATCH 05/34] Use /nonexistent rather than /dev/enoent Co-authored-by: Josh Triplett --- text/0000-command-ergonomics.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/text/0000-command-ergonomics.md b/text/0000-command-ergonomics.md index f13154c42eb..03d00d8dcdb 100644 --- a/text/0000-command-ergonomics.md +++ b/text/0000-command-ergonomics.md @@ -57,17 +57,17 @@ The following incorrect program fragments are all accepted today and run to comp ```rust Command::new("touch") - .args(&["/dev/enoent/touch-1"]); + .args(&["/nonexistent/touch-1"]); // ^ programmer surely wanted to actually run the command Command::new("touch") - .args(&["/dev/enoent/touch-2"]) + .args(&["/nonexistent/touch-2"]) .spawn()?; // ^ accidentally failed to wait, if programmer wanted to make a daemon // or zombie or something they should have to write let _ =. Command::new("touch") - .args(&["/dev/enoent/touch-3"]) + .args(&["/nonexistent/touch-3"]) .spawn()? .wait()?; // ^ accidentally failed to check exit status From 71583a40bf34eab13864be3451c51f0340ec0b5f Mon Sep 17 00:00:00 2001 From: Ian Jackson Date: Fri, 13 Jan 2023 13:03:23 +0000 Subject: [PATCH 06/34] get_output gives one string, not a Vec Co-authored-by: Josh Triplett --- text/0000-command-ergonomics.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-command-ergonomics.md b/text/0000-command-ergonomics.md index 03d00d8dcdb..de037b98a58 100644 --- a/text/0000-command-ergonomics.md +++ b/text/0000-command-ergonomics.md @@ -135,7 +135,7 @@ We aim to serve well each of the following people: After the child indicates EOF on its stdout, we will wait for it to finish and check the exit status. - * `fn get_output(&mut self) -> Result, SubprocessError>`: + * `fn get_output(&mut self) -> Result`: Runs the command and collects its stdout. Decodes the stdout as UTF-8, and fails if that's not possible. From 30a0106fed94ebfa81143ee17c2c283a795cc5e7 Mon Sep 17 00:00:00 2001 From: Ian Jackson Date: Fri, 13 Jan 2023 13:04:10 +0000 Subject: [PATCH 07/34] get_output_line gives one string, not a Vec Co-authored-by: Josh Triplett --- text/0000-command-ergonomics.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-command-ergonomics.md b/text/0000-command-ergonomics.md index de037b98a58..bdd6ec327e2 100644 --- a/text/0000-command-ergonomics.md +++ b/text/0000-command-ergonomics.md @@ -141,7 +141,7 @@ We aim to serve well each of the following people: Decodes the stdout as UTF-8, and fails if that's not possible. Does not trim any trailing line ending. - * `fn get_output_line(&mut self) -> Result, SubprocessError>`: + * `fn get_output_line(&mut self) -> Result`: Runs the command and collects its stdout. Decodes the stdout as UTF-8, and fails if that's not possible. From 36b59ca7231e02f54b6f1fcf3f78790061707b8d Mon Sep 17 00:00:00 2001 From: Ian Jackson Date: Fri, 13 Jan 2023 13:05:00 +0000 Subject: [PATCH 08/34] Fix typo Co-authored-by: Josh Triplett --- text/0000-command-ergonomics.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-command-ergonomics.md b/text/0000-command-ergonomics.md index bdd6ec327e2..f331f113322 100644 --- a/text/0000-command-ergonomics.md +++ b/text/0000-command-ergonomics.md @@ -395,7 +395,7 @@ for process invocation and output handling. ## Printing command arguments in `impl Display for SubprocessError` Perhaps printing the command arguments is overly verbose, -and we should print onliy the command name. +and we should print only the command name. ## `get_output` vs `read_output` naming. From 57f63d77236d77dc8ab07e00faddced08189e76b Mon Sep 17 00:00:00 2001 From: Ian Jackson Date: Fri, 13 Jan 2023 17:02:19 +0000 Subject: [PATCH 09/34] Add lint suggestion to stderr handling default Co-authored-by: Josh Triplett --- text/0000-command-ergonomics.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/text/0000-command-ergonomics.md b/text/0000-command-ergonomics.md index f331f113322..f85895392d2 100644 --- a/text/0000-command-ergonomics.md +++ b/text/0000-command-ergonomics.md @@ -432,6 +432,8 @@ Options are: to all the new methods. These seem unpalatably unergonomic. +5. One of the above, and additionally provide a lint that makes a best-effort attempt to catch this at compile time. + Here we propose option 1: treat as `inherit`. # Future possibilities From bbd287d37a508768b29817b7152bba63f0a5e5c1 Mon Sep 17 00:00:00 2001 From: Ian Jackson Date: Fri, 13 Jan 2023 16:44:57 +0000 Subject: [PATCH 10/34] Make SubprocessError be a transparent struct --- text/0000-command-ergonomics.md | 90 ++++++++++++++++++++++----------- 1 file changed, 61 insertions(+), 29 deletions(-) diff --git a/text/0000-command-ergonomics.md b/text/0000-command-ergonomics.md index f85895392d2..8c5684a30c5 100644 --- a/text/0000-command-ergonomics.md +++ b/text/0000-command-ergonomics.md @@ -220,33 +220,77 @@ with `piped` we capture that, and represent both the stderr and the exit status as problems within the `SubprocessError`. ```rust -impl SubprocessError { +/// Problem(s) which occurred while running a subprocess +/// +/// This struct represents problems which occurred while +/// running a subprocess. +/// +/// Running a subprocess is complex, and it is even possible for a single invocation +/// to give rise to more than one problem. +/// So this struct can contain zero or more such problems, +/// along with information about what was run, for error reporting. +/// +/// ### Tolerating certain kinds of error +/// +/// Don't check the fields of this struct one by one. +/// Future language revisions may add more fields representing new kinds of problem! +/// +/// Instead: +/// +/// * If you want to capture stderr but combine it with stdout, +/// TODO need way to combine them! +/// +/// * If you wish to tolerate other particular kind(s) of problem, +/// set the field for the problems you want to tolerate to `None` +/// (doing any ncecessary checks on the the existing values, +/// to see if it's really something you want to ignore). +/// Then call `.has_problem()`. +#[must_use] +#[non_exhaustive] +struct SubprocessError { /// The program, if we know it. - fn program(&self) -> Option<&OsStr>; + // + // Used in the `Display` impl so we get good error messages. + program: Option, /// The arguments, if we know them. - fn args(&self) -> Option>; + // + // Used in the `Display` impl so we get good error messages. + args: Vec, /// If the stdout was captured in memory, the stdout data. - fn stdout_bytes(&self) -> Option<&[u8]>; + // + // Needed so that a caller can have the output even if the program failed. + stdout_bytes: Option>, /// If the process exited and we collected its status, the exit status. - fn status(&self) -> Option; + /// + /// If this is present and not success, it is treated as a problem. + status: Option, - /// If trouble included nonempty stderr, the captured stderr - fn stderr_bytes(&self) -> Option<&[u8]>; + /// If the stderr was captured in memory, the stdout data. + /// + /// If this is present and nonempty, it is treated as a problem. + stderr_bytes: Option>, - /// If trouble included failure to spawn, the spawn error. - fn spawn_error() -> Option<&io::Error>; + /// If had a problem spawning, the spawn error. + spawn_error: Option, - /// If trouble included failure to talk to the child, the IO error. + /// If we had a problem talking to the child, the IO error. /// /// This might include problems which might be caused by child /// misbehaviour. - fn communication_error() -> Option<&io::Error>; + communication_error: Option, - /// If trouble included failed UTF-8 conversion. - fn utf8_error(&self) -> Option<&std::str::FromUtf8Error>; + /// If we had a problem converting stdout to UTF-8. + /// + /// The `error_len()` and `valid_up_to()` reference positions in `stdout_bytes`. + utf8_error: Option, +} +impl Debug for SubprocessError { + // print all the fields except `stdout_bytes`. +} +impl Error for SubprocessError { } ``` @@ -286,16 +330,6 @@ impl SubprocessError { // If we keep ExitStatusError fn from_exit_status_error(status: ExitStatusError) -> Self { } - fn set_program(&mut self, impl Into); - fn set_args(&mut self, impl IntoIterator>); - fn set_stdout_bytes(output: Option>>); - - fn set_status(&mut self, status: ExitStatus); - fn set_stderr_bytes(&mut self, stderr: impl Into>); - fn set_spawn_error(&mut self, error: Option); - fn set_communication_error(&mut self, error: Option); - fn set_utf8_error(&mut self, error: Option); - /// Find out if this error contains any actual error information /// /// Returns `false` for a fresh blank error, @@ -360,12 +394,10 @@ Alternatives and prior proposals include: but got bogged down due to lack of consensus on overall direction and some bikeshed issues. - * The `SubprocessError` type could be a - transparent non-exhaustive struct. - This might be reasonable, - since it's really just a bag of information - with getters and setters for every field. - But transparent structs are unfashionable in modern Rust. + * The `SubprocessError` type could be opaque with getters and setters. + Transparent structs are unfashionable in modern Rust. + However, providing getters and setters obscures what's going on and + greatly enlarges the API surface. * Instead of a single `SubprocessError` type used everywhere, there could be a different error type for the different calls. From 79d0d25aa16007bc749f1c6a4013d0e3ea39a73b Mon Sep 17 00:00:00 2001 From: Ian Jackson Date: Fri, 13 Jan 2023 16:55:09 +0000 Subject: [PATCH 11/34] Promote `just_status` to current proposed, from future --- text/0000-command-ergonomics.md | 86 +++++++++------------------------ 1 file changed, 24 insertions(+), 62 deletions(-) diff --git a/text/0000-command-ergonomics.md b/text/0000-command-ergonomics.md index 8c5684a30c5..6ce0476602e 100644 --- a/text/0000-command-ergonomics.md +++ b/text/0000-command-ergonomics.md @@ -239,6 +239,8 @@ and the exit status as problems within the `SubprocessError`. /// /// * If you want to capture stderr but combine it with stdout, /// TODO need way to combine them! +/// +/// * If you wish to tolerate only nonzero exit status, call `.just_status()`. /// /// * If you wish to tolerate other particular kind(s) of problem, /// set the field for the problems you want to tolerate to `None` @@ -342,6 +344,13 @@ impl SubprocessError { // and a caller can't reimplement it without making assumptons // about `SubprocessError`'s contents. fn has_problem(&self) -> bool; + + /// Returns `Ok` if the only reason for the failure was a nonzero exit status. + /// Otherwise returns `self`. + //// + /// Use this if you want to to tolerate some exit statuses, + /// but still fail if there were other problems. + pub fn just_status(self) -> Result; } impl Default for SubprocessError { ... } impl Clone for SubprocessError { ... } // contained io:Errors are in Arcs @@ -468,6 +477,21 @@ Options are: Here we propose option 1: treat as `inherit`. +# Examples + +## Conveniently running `diff` + +```rust + let result = Command::new("diff") + .args(["before","after"]) + .stderr(Stdio::piped()) // optional, could just let it inherit + .run(); + let status = match result { + Ok(()) => 0, + Err(err) => err.just_status()?.code(), + }; +``` + # Future possibilities [future-possibilities]: #future-possibilities @@ -512,65 +536,3 @@ Options might include: [`read2`](https://github.com/rust-lang/cargo/blob/58a961314437258065e23cb6316dfc121d96fb71/crates/cargo-util/src/read2.rs) * Expect users who want this to use pipes by hand (perhaps with threads), or async. - -## More convenient way to run `diff` - -With the proposed API, -completely correctly running `diff(1)` would look a bit like this: - -```rust - let result = Command::new("diff") - .args(["before","after"]) - .run(); - let status = match result { - Ok(()) => 0, - Err(err) => { - let status = err.status(); - err.set_status(ExitStatusExt::from_raw(0)); - if err.has_problem() { - return Err(err); - } - status.code() - } - }; -``` - -This is doable but cumbersome. -A naive Dionysus is likely to write: - -```rust - let status = match result { - Ok(()) => 0, - Err(err) => { - if ! err.status().success() { - err.status().code() - } else { - return Err(err); - } - } - }; -``` - -As it happens, this is correct in the sense that it won't malfunction, -since actually `run()`, without piped stderr, -cannot produce a `SubprocessError` -containing a nonzero exit status *and* any other problem. -But in a more complex situation it might be wrong. - -Perhaps: -```rust -impl SubprocessError { - /// Returns `Ok` if the only reason for the failure was a nonzero exit status. Otherwise returns `self`. - //// - /// Use this if you want to to tolerate some exit statuses, but still fail if there were other problems. - pub fn just_status(self) -> Result; -} -``` - -Then Dionysus can write: -```rust - let status = match result { - Ok(()) => 0, - Err(err) => err.just_status()?.code(), - }; -``` From 84e4c783ae194369bdbfc7ffcdb6b235afefab82 Mon Sep 17 00:00:00 2001 From: Ian Jackson Date: Fri, 13 Jan 2023 16:59:03 +0000 Subject: [PATCH 12/34] Rename to read_stdout_* --- text/0000-command-ergonomics.md | 42 ++++++++++++++++++--------------- 1 file changed, 23 insertions(+), 19 deletions(-) diff --git a/text/0000-command-ergonomics.md b/text/0000-command-ergonomics.md index 6ce0476602e..cd229abe7a0 100644 --- a/text/0000-command-ergonomics.md +++ b/text/0000-command-ergonomics.md @@ -85,10 +85,10 @@ for running the command and collecting its output: ```rust impl Command { fn run(&mut self) -> Result<(), SubprocessError>; - fn get_output_bytes(&mut self) -> Result, SubprocessError>; - fn get_output(&mut self) -> Result; - fn get_output_line(&mut self) -> Result; - fn get_output_read(&mut self) -> impl std::io::Read; + fn read_stdout_bytes(&mut self) -> Result, SubprocessError>; + fn read_stdout(&mut self) -> Result; + fn read_stdout_line(&mut self) -> Result; + fn stdout_readable(&mut self) -> impl std::io::Read; } struct SubprocessError { ... } impl From for io::Error { ... } @@ -129,26 +129,26 @@ We aim to serve well each of the following people: Equivalent to `.spawn()` followed by `.status()`, but with better error handling. - * `fn get_output_bytes(&mut self) -> Result, SubprocessError>`: + * `fn read_stdout_bytes(&mut self) -> Result, SubprocessError>`: Runs the command and collects its stdout. After the child indicates EOF on its stdout, we will wait for it to finish and check the exit status. - * `fn get_output(&mut self) -> Result`: + * `fn read_stdout(&mut self) -> Result`: Runs the command and collects its stdout. Decodes the stdout as UTF-8, and fails if that's not possible. Does not trim any trailing line ending. - * `fn get_output_line(&mut self) -> Result`: + * `fn read_stdout_line(&mut self) -> Result`: Runs the command and collects its stdout. Decodes the stdout as UTF-8, and fails if that's not possible. Fails unless the output is a single line (with or without line ending). Trims the line ending (if any). - * `fn get_output_read(&mut self) -> std::process::ChildOutputStream` + * `fn read_stdout_read(&mut self) -> std::process::ChildOutputStream` (where `struct ChildOutputStream` implements `io::Read` and is `Send + Sync + 'static`). @@ -367,11 +367,11 @@ to run subprocesses and produce good error messages, we could omit this error type (perhaps using something like `ExitStatusError`). -Perhaps we don't need all the `get_output` variants, +Perhaps we don't need all the `read_stdout` variants, and could require Bob to write out the boilerplate or provide his own helper function. -Perhaps we don't need `get_output_read`. +Perhaps we don't need `read_stdout_read`. However, avoiding deadlocks when reading subprocess output, and also doing error checks properly, @@ -411,12 +411,12 @@ Alternatives and prior proposals include: * Instead of a single `SubprocessError` type used everywhere, there could be a different error type for the different calls. For example, `run()` could have a different error type to - `get_output()`, + `read_stdout()`, since `run` doesn't need to represent UTF-8 conversion errors. This would not be very in keeping with the rest of `std::process`, which tends to unified types with variation selected at runtime. There would still have to be *a* type as complex as `SubprocessError`, - since that's what `get_output_read` needs. + since that's what `read_stdout_read` needs. # Prior art [prior-art]: #prior-art @@ -438,17 +438,21 @@ for process invocation and output handling. Perhaps printing the command arguments is overly verbose, and we should print only the command name. -## `get_output` vs `read_output` naming. +## `read_stdout` etc. naming. We have `fs::read_to_string`. -Possible names (taking `get_output_bytes` as the example): +Possible names (taking `read_stdout_bytes` as the example): - * `get_output_bytes` (proposed in this RFC) - * `output_bytes` but `output` is alread taken for bad `Output`. - * `run_get_output_bytes` - * `run_output_bytes` - * `read_output_bytes` + * `read_stdout_bytes` (proposed in this RFC) + * `stdout_bytes` but `stdout` is alread taken. + * `get_stdout_bytes` + * `run_get_stdout_bytes` + * `run_stdout_bytes` + +It is difficult to convey everything that is needed in a short name. +In particular, all of these functions spawn the program, +and wait (at an appropriate point) for it to exit. ## stderr handling default From 92ef0e752dbb5586476d78e674c8b817f2d37910 Mon Sep 17 00:00:00 2001 From: Ian Jackson Date: Fri, 13 Jan 2023 16:59:31 +0000 Subject: [PATCH 13/34] Rename to ProcessError --- text/0000-command-ergonomics.md | 70 ++++++++++++++++----------------- 1 file changed, 35 insertions(+), 35 deletions(-) diff --git a/text/0000-command-ergonomics.md b/text/0000-command-ergonomics.md index cd229abe7a0..9b7d86344cd 100644 --- a/text/0000-command-ergonomics.md +++ b/text/0000-command-ergonomics.md @@ -84,14 +84,14 @@ for running the command and collecting its output: ```rust impl Command { - fn run(&mut self) -> Result<(), SubprocessError>; - fn read_stdout_bytes(&mut self) -> Result, SubprocessError>; - fn read_stdout(&mut self) -> Result; - fn read_stdout_line(&mut self) -> Result; + fn run(&mut self) -> Result<(), ProcessError>; + fn read_stdout_bytes(&mut self) -> Result, ProcessError>; + fn read_stdout(&mut self) -> Result; + fn read_stdout_line(&mut self) -> Result; fn stdout_readable(&mut self) -> impl std::io::Read; } -struct SubprocessError { ... } -impl From for io::Error { ... } +struct ProcessError { ... } +impl From for io::Error { ... } ``` The `.output()` function and `std::process::Output` @@ -123,25 +123,25 @@ We aim to serve well each of the following people: ## New methods on `Command`: - * `fn run(&mut self) -> Result<(), SubprocessError>`: + * `fn run(&mut self) -> Result<(), ProcessError>`: Runs the command. Equivalent to `.spawn()` followed by `.status()`, but with better error handling. - * `fn read_stdout_bytes(&mut self) -> Result, SubprocessError>`: + * `fn read_stdout_bytes(&mut self) -> Result, ProcessError>`: Runs the command and collects its stdout. After the child indicates EOF on its stdout, we will wait for it to finish and check the exit status. - * `fn read_stdout(&mut self) -> Result`: + * `fn read_stdout(&mut self) -> Result`: Runs the command and collects its stdout. Decodes the stdout as UTF-8, and fails if that's not possible. Does not trim any trailing line ending. - * `fn read_stdout_line(&mut self) -> Result`: + * `fn read_stdout_line(&mut self) -> Result`: Runs the command and collects its stdout. Decodes the stdout as UTF-8, and fails if that's not possible. @@ -182,7 +182,7 @@ If `stderr(Stdio::piped())`, these new functions all collect the child's stderr. Then, if the stderr output is nonempty, this is considered an error, -and reported in the `SubprocessError`. +and reported in the `ProcessError`. These functions *do not* wait for EOF on stderr. Rather, they wait for child process termination and expect that @@ -204,20 +204,20 @@ The implementation may involve a temporary file, or an in-memory buffer, or both. -## New `struct SubprocessError` +## New `struct ProcessError` This new struct is used as the error type for the new methods. It can represents zero or more of the various distinct problems that can occur while running a process. -A `SubprocessError` returned by a `std` function will always +A `ProcessError` returned by a `std` function will always represent at least one problem (unless otherwise stated), but it may represent several For example a process which exited nonzero probably printed to stderr; with `piped` we capture that, and represent both the stderr -and the exit status as problems within the `SubprocessError`. +and the exit status as problems within the `ProcessError`. ```rust /// Problem(s) which occurred while running a subprocess @@ -249,7 +249,7 @@ and the exit status as problems within the `SubprocessError`. /// Then call `.has_problem()`. #[must_use] #[non_exhaustive] -struct SubprocessError { +struct ProcessError { /// The program, if we know it. // // Used in the `Display` impl so we get good error messages. @@ -289,10 +289,10 @@ struct SubprocessError { /// The `error_len()` and `valid_up_to()` reference positions in `stdout_bytes`. utf8_error: Option, } -impl Debug for SubprocessError { +impl Debug for ProcessError { // print all the fields except `stdout_bytes`. } -impl Error for SubprocessError { +impl Error for ProcessError { } ``` @@ -301,28 +301,28 @@ including the command's arguments. The arguments will be escaped or quoted in some way that renders a resulting error message unambiguous. -### `impl From for io::Error` +### `impl From for io::Error` -`SubprocessError` must be convertible to `io::Error` +`ProcessError` must be convertible to `io::Error` so that we can use it in `ChildOutputStream`'s `Read` implementation. This may also be convenient elsewhere. -The `io::ErrorKind` for a `SubprocessError` will be: +The `io::ErrorKind` for a `ProcessError` will be: * The `io::ErrorKind` from the spawn error, if any. * Otherwise, a new kind `io::ErrorKind::ProcessFailed`, which means that the subprocess itself failed. -### Further necessary APIs for `SubprocessError` +### Further necessary APIs for `ProcessError` We also provide ways for this new error to be constructed, which will be needed by other lower level libraries besides std, notably async frameworks: ```rust -impl SubprocessError { +impl ProcessError { /// Makes a "blank" error which doesn't contain any useful information /// /// `has_problem()` will return `false` until one of the setters @@ -342,7 +342,7 @@ impl SubprocessError { // We must provide this because it's needed for handling programs // with unusual exit status conventions (eg `diff(1)`) // and a caller can't reimplement it without making assumptons - // about `SubprocessError`'s contents. + // about `ProcessError`'s contents. fn has_problem(&self) -> bool; /// Returns `Ok` if the only reason for the failure was a nonzero exit status. @@ -350,10 +350,10 @@ impl SubprocessError { //// /// Use this if you want to to tolerate some exit statuses, /// but still fail if there were other problems. - pub fn just_status(self) -> Result; + pub fn just_status(self) -> Result; } -impl Default for SubprocessError { ... } -impl Clone for SubprocessError { ... } // contained io:Errors are in Arcs +impl Default for ProcessError { ... } +impl Clone for ProcessError { ... } // contained io:Errors are in Arcs ``` # Drawbacks @@ -361,7 +361,7 @@ impl Clone for SubprocessError { ... } // contained io:Errors are in Arcs This is nontrivial new API surface. -Much of the new API surface is in `SubprocessError`. +Much of the new API surface is in `ProcessError`. If we didn't want to try to make it easy for Rust programmers to run subprocesses and produce good error messages, we could omit this error type @@ -403,19 +403,19 @@ Alternatives and prior proposals include: but got bogged down due to lack of consensus on overall direction and some bikeshed issues. - * The `SubprocessError` type could be opaque with getters and setters. + * The `ProcessError` type could be opaque with getters and setters. Transparent structs are unfashionable in modern Rust. However, providing getters and setters obscures what's going on and greatly enlarges the API surface. - * Instead of a single `SubprocessError` type used everywhere, + * Instead of a single `ProcessError` type used everywhere, there could be a different error type for the different calls. For example, `run()` could have a different error type to `read_stdout()`, since `run` doesn't need to represent UTF-8 conversion errors. This would not be very in keeping with the rest of `std::process`, which tends to unified types with variation selected at runtime. - There would still have to be *a* type as complex as `SubprocessError`, + There would still have to be *a* type as complex as `ProcessError`, since that's what `read_stdout_read` needs. # Prior art @@ -433,7 +433,7 @@ for process invocation and output handling. # Unresolved questions [unresolved-questions]: #unresolved-questions -## Printing command arguments in `impl Display for SubprocessError` +## Printing command arguments in `impl Display for ProcessError` Perhaps printing the command arguments is overly verbose, and we should print only the command name. @@ -505,20 +505,20 @@ Possibilities include: * Abolish `ExitStatusError` * Stabilise `ExitStatusError` as-is - * `impl From for SubprocessError` + * `impl From for ProcessError` * `impl From for io::Error` Error messages from `ExitStatusError` are rather poor, -and it is likely that `SubprocessError` will +and it is likely that `ProcessError` will subsume most of its use cases. ## Async ecosystem could mirror these APIs * An async versions of `run()` seems like it would be convenient. * Async versions of the output-capturing runners too. - * Async frameworks ought to (be able to) use `SubprocessError`. + * Async frameworks ought to (be able to) use `ProcessError`. -Maybe `SubprocessError` would have to be able to contain a nested +Maybe `ProcessError` would have to be able to contain a nested `Arc`. That doesn't need to happen now. But it is one reason why `.has_problem()` needs to exist. From d6968208e9f91c74e9e09fe187902420fd9a6782 Mon Sep 17 00:00:00 2001 From: Ian Jackson Date: Fri, 13 Jan 2023 17:00:49 +0000 Subject: [PATCH 14/34] Document behaviour of read_stdout_line with no output at all --- text/0000-command-ergonomics.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/text/0000-command-ergonomics.md b/text/0000-command-ergonomics.md index 9b7d86344cd..298d18fe464 100644 --- a/text/0000-command-ergonomics.md +++ b/text/0000-command-ergonomics.md @@ -147,6 +147,8 @@ We aim to serve well each of the following people: Decodes the stdout as UTF-8, and fails if that's not possible. Fails unless the output is a single line (with or without line ending). Trims the line ending (if any). + If program prints no output at all, returns an empty string + (and this cannot be distinguished from the program printing just a newline). * `fn read_stdout_read(&mut self) -> std::process::ChildOutputStream` (where `struct ChildOutputStream` implements `io::Read` From 024e8cac66a76873ffbafb8b241cd11f400e0a83 Mon Sep 17 00:00:00 2001 From: Ian Jackson Date: Fri, 13 Jan 2023 17:04:48 +0000 Subject: [PATCH 15/34] Cross-reference for EOF and exit wait of read_stdout{,_line} --- text/0000-command-ergonomics.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/text/0000-command-ergonomics.md b/text/0000-command-ergonomics.md index 298d18fe464..665618ced05 100644 --- a/text/0000-command-ergonomics.md +++ b/text/0000-command-ergonomics.md @@ -137,13 +137,13 @@ We aim to serve well each of the following people: * `fn read_stdout(&mut self) -> Result`: - Runs the command and collects its stdout. + Runs the command and collects its stdout, as for `read_stdout_bytes`. Decodes the stdout as UTF-8, and fails if that's not possible. Does not trim any trailing line ending. * `fn read_stdout_line(&mut self) -> Result`: - Runs the command and collects its stdout. + Runs the command and collects its stdout, as for `read_stdout_bytes`. Decodes the stdout as UTF-8, and fails if that's not possible. Fails unless the output is a single line (with or without line ending). Trims the line ending (if any). From 07ffa9f9292c3ff85ac6c8628976b534bbc3daec Mon Sep 17 00:00:00 2001 From: Ian Jackson Date: Fri, 13 Jan 2023 17:06:22 +0000 Subject: [PATCH 16/34] Rename to ChildOutputStream --- text/0000-command-ergonomics.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/text/0000-command-ergonomics.md b/text/0000-command-ergonomics.md index 665618ced05..ab6e541c884 100644 --- a/text/0000-command-ergonomics.md +++ b/text/0000-command-ergonomics.md @@ -88,7 +88,7 @@ impl Command { fn read_stdout_bytes(&mut self) -> Result, ProcessError>; fn read_stdout(&mut self) -> Result; fn read_stdout_line(&mut self) -> Result; - fn stdout_readable(&mut self) -> impl std::io::Read; + fn stdout_reader(&mut self) -> impl std::io::Read; } struct ProcessError { ... } impl From for io::Error { ... } @@ -150,13 +150,13 @@ We aim to serve well each of the following people: If program prints no output at all, returns an empty string (and this cannot be distinguished from the program printing just a newline). - * `fn read_stdout_read(&mut self) -> std::process::ChildOutputStream` - (where `struct ChildOutputStream` implements `io::Read` + * `fn stdout_reader(&mut self) -> std::process::ChildOutputReader` + (where `struct ChildOutputReader` implements `io::Read` and is `Send + Sync + 'static`). Starts the command, allowing the caller to read the stdout in a streaming way. - Neither EOF nor an error will be reported by `ChildOutputStream` + Neither EOF nor an error will be reported by `ChildOutputReader` until the child has exited, *and* the stdout pipe reports EOF. (This includes errors due to nonempty stderr, if stderr was set to `piped`.) @@ -306,7 +306,7 @@ a resulting error message unambiguous. ### `impl From for io::Error` `ProcessError` must be convertible to `io::Error` -so that we can use it in `ChildOutputStream`'s +so that we can use it in `ChildOutputReader`'s `Read` implementation. This may also be convenient elsewhere. From 8e96f69cadf0181bf07273ef96ddd952d8bb4d00 Mon Sep 17 00:00:00 2001 From: Ian Jackson Date: Fri, 13 Jan 2023 17:10:44 +0000 Subject: [PATCH 17/34] Move deprecation of .output() to Future possibilities --- text/0000-command-ergonomics.md | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/text/0000-command-ergonomics.md b/text/0000-command-ergonomics.md index ab6e541c884..a027a9d8a81 100644 --- a/text/0000-command-ergonomics.md +++ b/text/0000-command-ergonomics.md @@ -94,9 +94,6 @@ struct ProcessError { ... } impl From for io::Error { ... } ``` -The `.output()` function and `std::process::Output` -will be deprecated. - No significant changes are made to the `Command` construction APIs, but it may become necessary to call `.stderr()` explicitly to use the new methods (see Unresolved Questions). @@ -169,10 +166,6 @@ which people were previously required to use. ## Deprecations - * Deprecate `std::process::Command::output()`. - This API cannot be fixed; - see [#73126](https://github.com/rust-lang/rust/issues/73126). - * Apply `#[must_use]` to `Command` and `Child`. * Apply `#[must_use]` to `ExitStatus`. @@ -501,6 +494,16 @@ Here we propose option 1: treat as `inherit`. # Future possibilities [future-possibilities]: #future-possibilities +## Deprecating `Command.output()` and `std::process::Output` + +The `.output()` and `Output` API has an error handling hazard, +see [#73126](https://github.com/rust-lang/rust/issues/73126). + +Perhaps it should be deprecated at some point. + +However, it is a popular API so that would be disruptive, +and some things are easier to do with `Output` than with `Result<..., ProcessError>`. + ## Changes to `ExitStatusError` Possibilities include: From cd1c96df365552c2304ec0a79421da988c05e940 Mon Sep 17 00:00:00 2001 From: Ian Jackson Date: Fri, 13 Jan 2023 17:17:55 +0000 Subject: [PATCH 18/34] Document Error impl cause() behaviour --- text/0000-command-ergonomics.md | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/text/0000-command-ergonomics.md b/text/0000-command-ergonomics.md index a027a9d8a81..b66b994eb52 100644 --- a/text/0000-command-ergonomics.md +++ b/text/0000-command-ergonomics.md @@ -288,6 +288,7 @@ impl Debug for ProcessError { // print all the fields except `stdout_bytes`. } impl Error for ProcessError { + // fn cause() always returns None. } ``` @@ -433,6 +434,29 @@ for process invocation and output handling. Perhaps printing the command arguments is overly verbose, and we should print only the command name. +## `cause` in `Error` impl of `ProcessError` + +Because maybe several things went wrong, ever providing a `Some` `cause` +would involve prioritising multiple possible problems. + +Also, (IMO doubtful) EHWG guidelines about `Display` implementations +say that we shouldn't include information about the cause in our own `Display`. + +This leaves us with the following options: + + 1. Not include the actual thing that went wrong in our `Display`. + This would in practice result in very poor error messages from many + natural uses of this API. + Also whether problem A appears in the `Display` output might depend + on whether "higher-priority cause" B is present. + This seems madness. + + 2. Violate the EHWG guideline. + (Or try to get it deprecated.) + + 3. Not include a `cause` at all. + This is the option we propose. + ## `read_stdout` etc. naming. We have `fs::read_to_string`. From 80f14966dc9cd4fb7409e0aa2064bbe9f4433343 Mon Sep 17 00:00:00 2001 From: Ian Jackson Date: Fri, 13 Jan 2023 17:23:10 +0000 Subject: [PATCH 19/34] Discuss sensitive command line arguments --- text/0000-command-ergonomics.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/text/0000-command-ergonomics.md b/text/0000-command-ergonomics.md index b66b994eb52..5ec3ad2bc0e 100644 --- a/text/0000-command-ergonomics.md +++ b/text/0000-command-ergonomics.md @@ -434,6 +434,12 @@ for process invocation and output handling. Perhaps printing the command arguments is overly verbose, and we should print only the command name. +Sometimes people pass sensitive information (such ass passwords) in command line arguments. +This is not a good idea, because command line arguments are generally public on Unix. + +Perhaps some option could be added in the future to control this. +For now, we propose always printing the arguments. + ## `cause` in `Error` impl of `ProcessError` Because maybe several things went wrong, ever providing a `Some` `cause` From 9d8a2a4a7951fffbb21f3d0fd833f5d0dc82f657 Mon Sep 17 00:00:00 2001 From: Ian Jackson Date: Fri, 13 Jan 2023 17:35:46 +0000 Subject: [PATCH 20/34] Discuss combine-and-interleave of stdout and stderr --- text/0000-command-ergonomics.md | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) diff --git a/text/0000-command-ergonomics.md b/text/0000-command-ergonomics.md index 5ec3ad2bc0e..d8307c444a4 100644 --- a/text/0000-command-ergonomics.md +++ b/text/0000-command-ergonomics.md @@ -524,6 +524,39 @@ Here we propose option 1: treat as `inherit`. # Future possibilities [future-possibilities]: #future-possibilities +## Providing a way to combine and interleave stdout and stderr + +Currently, `Command` insists on separating out stdout and stderr, +if you ask to capture them. +If you want them combined +(which is the only way to preserve the relative ordering) +you must do one of: + + * run a command which itself does the redirection + (easy using the shell on Unix) + + * send them each to your own stdout/stderr with `inherit` + and expect your caller to combine them + + * sent themk to the *same* one of your stdout/stderr + which will be possible after + https://github.com/rust-lang/rust/pull/88561) + +We should provide something like this: + +``` +impl Command { + /// Arranges that the command's stderr will be sent to wherever its stdout is going + fn stderr_to_stdout(); +} +``` + +(It is not sensibly possible at least on Unix +to get all of the stdout and stderr output +and find out *both* what order it out came in, +*and* which data was printed to which stream. +This is a limitation of the POSIX APIs.) + ## Deprecating `Command.output()` and `std::process::Output` The `.output()` and `Output` API has an error handling hazard, From e16f199428af1b96cec73654b375a3de040449dc Mon Sep 17 00:00:00 2001 From: Ian Jackson Date: Fri, 13 Jan 2023 17:40:23 +0000 Subject: [PATCH 21/34] Use read_stdout for the bytes version --- text/0000-command-ergonomics.md | 31 ++++++++++++++++++------------- 1 file changed, 18 insertions(+), 13 deletions(-) diff --git a/text/0000-command-ergonomics.md b/text/0000-command-ergonomics.md index d8307c444a4..65bebe547b8 100644 --- a/text/0000-command-ergonomics.md +++ b/text/0000-command-ergonomics.md @@ -85,8 +85,8 @@ for running the command and collecting its output: ```rust impl Command { fn run(&mut self) -> Result<(), ProcessError>; - fn read_stdout_bytes(&mut self) -> Result, ProcessError>; - fn read_stdout(&mut self) -> Result; + fn read_stdout(&mut self) -> Result, ProcessError>; + fn read_stdout_string(&mut self) -> Result; fn read_stdout_line(&mut self) -> Result; fn stdout_reader(&mut self) -> impl std::io::Read; } @@ -126,21 +126,21 @@ We aim to serve well each of the following people: Equivalent to `.spawn()` followed by `.status()`, but with better error handling. - * `fn read_stdout_bytes(&mut self) -> Result, ProcessError>`: + * `fn read_stdout(&mut self) -> Result, ProcessError>`: Runs the command and collects its stdout. After the child indicates EOF on its stdout, we will wait for it to finish and check the exit status. - * `fn read_stdout(&mut self) -> Result`: + * `fn read_stdout_string(&mut self) -> Result`: - Runs the command and collects its stdout, as for `read_stdout_bytes`. + Runs the command and collects its stdout, as for `read_stdout`. Decodes the stdout as UTF-8, and fails if that's not possible. Does not trim any trailing line ending. * `fn read_stdout_line(&mut self) -> Result`: - Runs the command and collects its stdout, as for `read_stdout_bytes`. + Runs the command and collects its stdout, as for `read_stdout`. Decodes the stdout as UTF-8, and fails if that's not possible. Fails unless the output is a single line (with or without line ending). Trims the line ending (if any). @@ -367,7 +367,7 @@ Perhaps we don't need all the `read_stdout` variants, and could require Bob to write out the boilerplate or provide his own helper function. -Perhaps we don't need `read_stdout_read`. +Perhaps we don't need `stdout_reader`. However, avoiding deadlocks when reading subprocess output, and also doing error checks properly, @@ -467,18 +467,23 @@ This leaves us with the following options: We have `fs::read_to_string`. -Possible names (taking `read_stdout_bytes` as the example): +Possible names (taking `read_stdout_string` as the example): - * `read_stdout_bytes` (proposed in this RFC) - * `stdout_bytes` but `stdout` is alread taken. - * `get_stdout_bytes` - * `run_get_stdout_bytes` - * `run_stdout_bytes` + * `read_stdout_string` (proposed in this RFC) + * `stdout_string` but `stdout` is alread taken. + * `get_stdout_string` + * `run_get_stdout_string` + * `run_stdout_string` It is difficult to convey everything that is needed in a short name. In particular, all of these functions spawn the program, and wait (at an appropriate point) for it to exit. +Should `read_output` be the one that returns `Vec` +or the one that returns `Vec` ? +Precedent in the stdlib is usually to have the bytes version undecorated +(eg, `fs::read_to_string`). + ## stderr handling default What should happen if the Rust programmer didn't call `.stderr()` ? From 92ee567a6367ccd39fc7881a2853518ab8a4a94c Mon Sep 17 00:00:00 2001 From: Ian Jackson Date: Fri, 13 Jan 2023 17:50:49 +0000 Subject: [PATCH 22/34] Move read_stdout_line to Future possibilities --- text/0000-command-ergonomics.md | 26 ++++++++++++++++---------- 1 file changed, 16 insertions(+), 10 deletions(-) diff --git a/text/0000-command-ergonomics.md b/text/0000-command-ergonomics.md index 65bebe547b8..4cd3d01c6de 100644 --- a/text/0000-command-ergonomics.md +++ b/text/0000-command-ergonomics.md @@ -87,7 +87,6 @@ impl Command { fn run(&mut self) -> Result<(), ProcessError>; fn read_stdout(&mut self) -> Result, ProcessError>; fn read_stdout_string(&mut self) -> Result; - fn read_stdout_line(&mut self) -> Result; fn stdout_reader(&mut self) -> impl std::io::Read; } struct ProcessError { ... } @@ -138,15 +137,6 @@ We aim to serve well each of the following people: Decodes the stdout as UTF-8, and fails if that's not possible. Does not trim any trailing line ending. - * `fn read_stdout_line(&mut self) -> Result`: - - Runs the command and collects its stdout, as for `read_stdout`. - Decodes the stdout as UTF-8, and fails if that's not possible. - Fails unless the output is a single line (with or without line ending). - Trims the line ending (if any). - If program prints no output at all, returns an empty string - (and this cannot be distinguished from the program printing just a newline). - * `fn stdout_reader(&mut self) -> std::process::ChildOutputReader` (where `struct ChildOutputReader` implements `io::Read` and is `Send + Sync + 'static`). @@ -562,6 +552,22 @@ and find out *both* what order it out came in, *and* which data was printed to which stream. This is a limitation of the POSIX APIs.) +## Provide a way to read exactly a single line + +``` + * `fn read_stdout_line(&mut self) -> Result`: + + Runs the command and collects its stdout, as for `read_stdout`. + Decodes the stdout as UTF-8, and fails if that's not possible. + Fails unless the output is a single line (with or without line ending). + Trims the line ending (if any). + If program prints no output at all, returns an empty string + (and this cannot be distinguished from the program printing just a newline). +``` + +It's not clear if this ought to live here or +as a method on `String`. + ## Deprecating `Command.output()` and `std::process::Output` The `.output()` and `Output` API has an error handling hazard, From b8c623cd99bb70867ed7236caef96ad6965a13ba Mon Sep 17 00:00:00 2001 From: Ian Jackson Date: Fri, 13 Jan 2023 17:55:09 +0000 Subject: [PATCH 23/34] Make error fields pub --- text/0000-command-ergonomics.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/text/0000-command-ergonomics.md b/text/0000-command-ergonomics.md index 4cd3d01c6de..9fca296dca4 100644 --- a/text/0000-command-ergonomics.md +++ b/text/0000-command-ergonomics.md @@ -238,41 +238,41 @@ struct ProcessError { /// The program, if we know it. // // Used in the `Display` impl so we get good error messages. - program: Option, + pub program: Option, /// The arguments, if we know them. // // Used in the `Display` impl so we get good error messages. - args: Vec, + pub args: Vec, /// If the stdout was captured in memory, the stdout data. // // Needed so that a caller can have the output even if the program failed. - stdout_bytes: Option>, + pub stdout_bytes: Option>, /// If the process exited and we collected its status, the exit status. /// /// If this is present and not success, it is treated as a problem. - status: Option, + pub status: Option, /// If the stderr was captured in memory, the stdout data. /// /// If this is present and nonempty, it is treated as a problem. - stderr_bytes: Option>, + pub stderr_bytes: Option>, /// If had a problem spawning, the spawn error. - spawn_error: Option, + pub spawn_error: Option, /// If we had a problem talking to the child, the IO error. /// /// This might include problems which might be caused by child /// misbehaviour. - communication_error: Option, + pub communication_error: Option, /// If we had a problem converting stdout to UTF-8. /// /// The `error_len()` and `valid_up_to()` reference positions in `stdout_bytes`. - utf8_error: Option, + pub utf8_error: Option, } impl Debug for ProcessError { // print all the fields except `stdout_bytes`. From d94ad5ddae5c5f2ceea8dc57c9d3f9d79023eb05 Mon Sep 17 00:00:00 2001 From: Ian Jackson Date: Fri, 13 Jan 2023 17:55:23 +0000 Subject: [PATCH 24/34] Discuss status of the various ProcessError functions --- text/0000-command-ergonomics.md | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/text/0000-command-ergonomics.md b/text/0000-command-ergonomics.md index 9fca296dca4..dff6c7485da 100644 --- a/text/0000-command-ergonomics.md +++ b/text/0000-command-ergonomics.md @@ -301,11 +301,12 @@ The `io::ErrorKind` for a `ProcessError` will be: * Otherwise, a new kind `io::ErrorKind::ProcessFailed`, which means that the subprocess itself failed. -### Further necessary APIs for `ProcessError` +### Further APIs for `ProcessError` -We also provide ways for this new error to be constructed, -which will be needed by other lower level libraries besides std, -notably async frameworks: +A `ProcessError` is a transparent `Default` struct so it can be +constructed outside std, for example by async frameworks. + +We propose the following additional methods: ```rust impl ProcessError { @@ -313,6 +314,9 @@ impl ProcessError { /// /// `has_problem()` will return `false` until one of the setters /// is used to store an actual problem. + // + // This is a name for the `Default` impl, and not essential, + // although it's conventional in Rust to provide it. fn new_empty() -> Self { } // If we keep ExitStatusError @@ -336,6 +340,10 @@ impl ProcessError { //// /// Use this if you want to to tolerate some exit statuses, /// but still fail if there were other problems. + // + // This is optional, and could be a separate feature from the rest of the RFC. + // But it does make running programs like `diff` considerably easier. + // (It is also implementable externally in terms of .has_problem()`.) pub fn just_status(self) -> Result; } impl Default for ProcessError { ... } From ac61312b80ab7a29aa129a4c1687bba0edfbcd4f Mon Sep 17 00:00:00 2001 From: Ian Jackson Date: Fri, 13 Jan 2023 18:00:08 +0000 Subject: [PATCH 25/34] Make SubprocessError not be Clone --- text/0000-command-ergonomics.md | 1 - 1 file changed, 1 deletion(-) diff --git a/text/0000-command-ergonomics.md b/text/0000-command-ergonomics.md index dff6c7485da..337dccbe5ec 100644 --- a/text/0000-command-ergonomics.md +++ b/text/0000-command-ergonomics.md @@ -347,7 +347,6 @@ impl ProcessError { pub fn just_status(self) -> Result; } impl Default for ProcessError { ... } -impl Clone for ProcessError { ... } // contained io:Errors are in Arcs ``` # Drawbacks From 732e555f619e7706544f5dc74409a91fffa5c441 Mon Sep 17 00:00:00 2001 From: Ian Jackson Date: Fri, 13 Jan 2023 18:11:07 +0000 Subject: [PATCH 26/34] Propose adding a docs warning to Command.output() --- text/0000-command-ergonomics.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/text/0000-command-ergonomics.md b/text/0000-command-ergonomics.md index 337dccbe5ec..b38d5ee0729 100644 --- a/text/0000-command-ergonomics.md +++ b/text/0000-command-ergonomics.md @@ -161,6 +161,12 @@ which people were previously required to use. * Apply `#[must_use]` to `ExitStatus`. (May require fixing quite a few of the examples.) + * Add a warning to the docs for `Command.output()` about the lost error bugs, + in particular the need to check `.status` and the lack of any compiler + warning if one doesn't. Suggest to the reader to consider + `.run()` or `.read_stdout*` instead. + Do not deprecate `.output()` though. + ## stderr handling If `stderr(Stdio::piped())`, From 2292b56887cdcef06dd3295c814cda98c74224c6 Mon Sep 17 00:00:00 2001 From: Ian Jackson Date: Thu, 19 Jan 2023 12:57:37 +0000 Subject: [PATCH 27/34] Fix typo Co-authored-by: Josh Triplett --- text/0000-command-ergonomics.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-command-ergonomics.md b/text/0000-command-ergonomics.md index b38d5ee0729..628edd5d4ad 100644 --- a/text/0000-command-ergonomics.md +++ b/text/0000-command-ergonomics.md @@ -546,7 +546,7 @@ you must do one of: * send them each to your own stdout/stderr with `inherit` and expect your caller to combine them - * sent themk to the *same* one of your stdout/stderr + * send them to the *same* one of your stdout/stderr which will be possible after https://github.com/rust-lang/rust/pull/88561) From 9167ef5ea793121dd83afbd003128b28e2ec5836 Mon Sep 17 00:00:00 2001 From: Ian Jackson Date: Thu, 19 Jan 2023 13:09:19 +0000 Subject: [PATCH 28/34] Expand on reasoning for ProcessError::Default Co-authored-by: Josh Triplett --- text/0000-command-ergonomics.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/text/0000-command-ergonomics.md b/text/0000-command-ergonomics.md index 628edd5d4ad..ef5899d7778 100644 --- a/text/0000-command-ergonomics.md +++ b/text/0000-command-ergonomics.md @@ -321,8 +321,9 @@ impl ProcessError { /// `has_problem()` will return `false` until one of the setters /// is used to store an actual problem. // - // This is a name for the `Default` impl, and not essential, - // although it's conventional in Rust to provide it. + // This is equivalent to `ProcessError::default()`, but provides + // a more semantically meaningful name, making it clear that it + // returns an empty error that needs filling in. fn new_empty() -> Self { } // If we keep ExitStatusError From 561cb3f8cf840955d0ebca8889651e6a9350781d Mon Sep 17 00:00:00 2001 From: Ian Jackson Date: Thu, 19 Jan 2023 13:10:39 +0000 Subject: [PATCH 29/34] Do a missed rename Co-authored-by: Josh Triplett --- text/0000-command-ergonomics.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/text/0000-command-ergonomics.md b/text/0000-command-ergonomics.md index ef5899d7778..b3cd3def9a3 100644 --- a/text/0000-command-ergonomics.md +++ b/text/0000-command-ergonomics.md @@ -483,7 +483,7 @@ It is difficult to convey everything that is needed in a short name. In particular, all of these functions spawn the program, and wait (at an appropriate point) for it to exit. -Should `read_output` be the one that returns `Vec` +Should `read_stdout` be the one that returns `Vec` or the one that returns `Vec` ? Precedent in the stdlib is usually to have the bytes version undecorated (eg, `fs::read_to_string`). From 01030129becc621b178c6f38073452cb8250971d Mon Sep 17 00:00:00 2001 From: Ian Jackson Date: Thu, 19 Jan 2023 13:04:53 +0000 Subject: [PATCH 30/34] Slightly weaken assertion about stdout/stderr demux --- text/0000-command-ergonomics.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/text/0000-command-ergonomics.md b/text/0000-command-ergonomics.md index b3cd3def9a3..1803d066374 100644 --- a/text/0000-command-ergonomics.md +++ b/text/0000-command-ergonomics.md @@ -560,8 +560,8 @@ impl Command { } ``` -(It is not sensibly possible at least on Unix -to get all of the stdout and stderr output +(It [can be difficult or impossible](https://docs.rs/io-mux/latest/io_mux/) at least on Unix +to reliably get all of the stdout and stderr output and find out *both* what order it out came in, *and* which data was printed to which stream. This is a limitation of the POSIX APIs.) From efa40c22aa3f82e550c5b3853d51e439f9dcad62 Mon Sep 17 00:00:00 2001 From: Ian Jackson Date: Thu, 19 Jan 2023 13:08:22 +0000 Subject: [PATCH 31/34] Add note about portability to comments about command lines --- text/0000-command-ergonomics.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/text/0000-command-ergonomics.md b/text/0000-command-ergonomics.md index 1803d066374..d3ff0bbfe99 100644 --- a/text/0000-command-ergonomics.md +++ b/text/0000-command-ergonomics.md @@ -439,7 +439,8 @@ Perhaps printing the command arguments is overly verbose, and we should print only the command name. Sometimes people pass sensitive information (such ass passwords) in command line arguments. -This is not a good idea, because command line arguments are generally public on Unix. +This is not a good idea in portable software, +because command line arguments are generally public on Unix. Perhaps some option could be added in the future to control this. For now, we propose always printing the arguments. From 37c4699ed4421384ece8ee29c0cc4ac14f2ff092 Mon Sep 17 00:00:00 2001 From: Ian Jackson Date: Thu, 19 Jan 2023 13:11:53 +0000 Subject: [PATCH 32/34] Expand on examples of why we want external ProcessError construction --- text/0000-command-ergonomics.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/text/0000-command-ergonomics.md b/text/0000-command-ergonomics.md index d3ff0bbfe99..c5ba3f83258 100644 --- a/text/0000-command-ergonomics.md +++ b/text/0000-command-ergonomics.md @@ -310,7 +310,8 @@ The `io::ErrorKind` for a `ProcessError` will be: ### Further APIs for `ProcessError` A `ProcessError` is a transparent `Default` struct so it can be -constructed outside std, for example by async frameworks. +constructed outside std, for example by async frameworks, +or other code which handles launching subprocesses. We propose the following additional methods: From 3f8db6e4d4e1a6ca190ac13fa6c041b7bc8e635a Mon Sep 17 00:00:00 2001 From: Ian Jackson Date: Thu, 19 Jan 2023 13:20:09 +0000 Subject: [PATCH 33/34] Introduce other_error replacing communication_error --- text/0000-command-ergonomics.md | 31 ++++++++++++++++++++++++------- 1 file changed, 24 insertions(+), 7 deletions(-) diff --git a/text/0000-command-ergonomics.md b/text/0000-command-ergonomics.md index c5ba3f83258..48af78ea076 100644 --- a/text/0000-command-ergonomics.md +++ b/text/0000-command-ergonomics.md @@ -269,11 +269,30 @@ struct ProcessError { /// If had a problem spawning, the spawn error. pub spawn_error: Option, - /// If we had a problem talking to the child, the IO error. + /// If we had some other problem, that error. + /// + /// This could a problem talking to the child, or collecting its exit status. /// /// This might include problems which might be caused by child /// misbehaviour. - pub communication_error: Option, + // + // In an earlier draft this was `communication_error: Option`. + // But an `io::Error` is not sufficient, because we would also want to report + // what it was we were trying to do that failed. + // + // Communication errors like this are going to be rare + // (at least, on Unix, I think they "can never happen" + // barring bugs in the kernel, stdlib or libc, + // unreasonable signal dispositions, + // UB or fd bugs in the Rust program, or the like). + // We don't want to expose lots of complicated details here, + // + // Also, these aren't usefully tolerable by applications. + // So a Box is good enough. + // + // Making it `other` allows it to be used by outside-stdlib + // constructors of process errors. + pub other_error: Option>, /// If we had a problem converting stdout to UTF-8. /// @@ -419,6 +438,9 @@ Alternatives and prior proposals include: There would still have to be *a* type as complex as `ProcessError`, since that's what `read_stdout_read` needs. + * Maybe `ProcessError::other_error` ought not to exist yet, + and we should have a separate `ProcessError::communication_error`. + # Prior art [prior-art]: #prior-art @@ -613,11 +635,6 @@ subsume most of its use cases. * Async versions of the output-capturing runners too. * Async frameworks ought to (be able to) use `ProcessError`. -Maybe `ProcessError` would have to be able to contain a nested -`Arc`. -That doesn't need to happen now. -But it is one reason why `.has_problem()` needs to exist. - ## More flexible and less synchronous output handling We could provide a more concurrent API, From a967e0f2dece265aec45da2af46b511d76479d6c Mon Sep 17 00:00:00 2001 From: Ian Jackson Date: Thu, 19 Jan 2023 13:24:12 +0000 Subject: [PATCH 34/34] Introduce other_error replacing communication_error (clarify) --- text/0000-command-ergonomics.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/text/0000-command-ergonomics.md b/text/0000-command-ergonomics.md index 48af78ea076..28401362084 100644 --- a/text/0000-command-ergonomics.md +++ b/text/0000-command-ergonomics.md @@ -277,8 +277,10 @@ struct ProcessError { /// misbehaviour. // // In an earlier draft this was `communication_error: Option`. - // But an `io::Error` is not sufficient, because we would also want to report - // what it was we were trying to do that failed. + // But an `io::Error` is not ideal, + // because we need to represent what we were doing, for reporting in messages, + // (so it would have to a custom type boxed inside the `io::Error` anyway) + // and the `io::ErrorKind` isn't very meaningful. // // Communication errors like this are going to be rare // (at least, on Unix, I think they "can never happen"