io: retry writes on interrupts#12008
Conversation
| if wwrite_to_fd(s, self.fd).is_none() { | ||
| // Some of our builtins emit multiple screens worth of data sent to a pager (the primary | ||
| // example being the `history` builtin) and receiving SIGINT should be considered normal and | ||
| // non-exceptional (user request to abort via Ctrl-C), meaning we shouldn't print an error. |
There was a problem hiding this comment.
Technically, the point about the history builtin should still be valid.
Maybe not for history in practice because we truncate it at 256k elements,
but I'd think that we should be able to cancel mid-output something like
fish -c 'set x "$(seq 2000000)"; echo $x'
I tried that and it's very unreliable; cancellation only works like 1/5 times at most,
maybe the signal is somehow not delivered fast enough
There was a problem hiding this comment.
there are two non-test uses of wwrite_to_fd
- the for flogging - for that one, retrying seems fine (a corrupted log file is no big deal but still better to avoid it)
- the one in FdOutputStream, which is instantiated for
- the fish_indent binary
- the fish_key_reader binary
- builtins writing to stdout with no redirections
- builtins writing to a file
- builtins writing to a pipe
It seems like this change is a good idea,
since it reduces corruption and we basically never actually write megabytes of data in a single wwrite_to_fd() call.
But we should probably move the ctrl-c cancellation logic to all the relevant users of FdOutputStream (i.e. builtins)
I wonder if we really want ctrl-c to potentially stop builtin output.
I guess cancellation works for external programs, so emulating that seems correct.
It means that ctrl-c can potentially cancel prompts, event handlers etc, which might be surprising,
but that's an unrelated issue.
builtin -n lists 68 builtins but most should not ever produce a lot of output.
So we can go through some of them.
Loops (while etc) are not relevant because fish already checks for cancellation at each loop iteration.
So it's mostly about builtins that loop internally.
Something like echo might be fine in practice because something like n=30 echo (seq $n)(seq $n)(seq $n)(seq $n) will simply error because the command substitution is too large.
It might be interesting to find out what's the maximum number of megabytes we can make fish pass to one wwrite_to_fd() call.
foo
But if that's too complicated we can squint and ignore it, or I can take a look later.
For things like the string commands, we might want to check for cancellation everytime after processing an argument?
I'm not saying we necessarily need to reimplement cancellation for everything today,
but we should keep in mind, we might want it eventually.
diff --git a/src/builtins/string/repeat.rs b/src/builtins/string/repeat.rs
index 5d540a6168..c10f6feee0 100644
--- a/src/builtins/string/repeat.rs
+++ b/src/builtins/string/repeat.rs
@@ -154,6 +154,8 @@
i -= chunk.len();
}
chunk.clear();
+ // TODO use SigChecker to check for cancellation. Not sure if we should stop mid-argument though.
+ // in case we have something like string repeat 10000000 foo\n
}
}
diff --git a/src/history.rs b/src/history.rs
index 2e9ed58ab6..f187ede7a9 100644
--- a/src/history.rs
+++ b/src/history.rs
@@ -1476,6 +1476,8 @@
}
if !streams.out.append(item) {
+ // TODO adjust the comment (we can no longer fail due to ctrl-c)
+ // but also use a SigChecker or equivalent here to exit early
// Don't force an error if output was aborted (typically via Ctrl-C/SIGINT); just don't
// try writing any more.
output_error = true;There was a problem hiding this comment.
I added code to address cancellation at the two locations in your diff. There are a lot more users of this code, but I think at least adding some examples of how cancellation can be handled with the new code is helpful.
There was a problem hiding this comment.
We can cancel this:
cat **.fish | command fish_indent
but not this (even without this change!)
cat **.fish | builtin fish_indent
not yet sure why,
but it seems like we should allow cancelling this.
I'm not sure what's the best, but an obvious way is the existing one:
allow to (at least optionally) have FdOutputStream abandon itself after EINTR.
It doesn't help that we fail to propagate the error here:
streams.out.append(bytes2wcstring(&colored_output));
we wrongly keep going after ctrl-c (i.e. we look at the next file).
Propagating this error everywhere where we could have a fd output stream might be a lot of work.
Interrupts should not prevent output from being completed, unless they
result in control flow not continuing where it was interrupted.
So given
echo foo
echo bar
if (hypothetically) the first one is interrupted, we don't print the second one AFAIK,
and as a consequence of your implication, the first echo statement should (or can?) actually exit early on EINTR.
I wonder if there is a nice way to make this true for all builtins.
When should ctrl-c not affect control flow?
I haven't checked other builtins yet.
This change is hard to prove correct, so I'd be careful about doing this.
| // Some of our builtins emit multiple screens worth of data sent to a pager (the primary | ||
| // example being the `history` builtin) and receiving SIGINT should be considered normal and | ||
| // non-exceptional (user request to abort via Ctrl-C), meaning we shouldn't print an error. | ||
| if errno::errno().0 == EINTR && self.sigcheck.check() { |
There was a problem hiding this comment.
It's a shame we can't use sigaction()'s SA_RESTART,
which would add automatic retrying to things like write(3)
but not to things that are supposed to be interruptible like poll(3) or sleep(3).
I guess Rust's stdlib does pretty much the same.
In future, if we add a slightly different event loop or a separate thread for handling input (and UI output),
then we could read ctrl-c from the TTY rather than relying on signals.
There was a problem hiding this comment.
So what do you think we should do with this PR? Merge it as is? Add some TODO comments? Come up with a better implementation?
| } | ||
| true | ||
| fn do_write(fd: RawFd, total_written: &mut usize, buf: &[u8]) -> std::io::Result<()> { | ||
| write_to_fd(buf, fd)?; |
There was a problem hiding this comment.
(unrelated) I guess write_to_fd is equivalent to std::fs::File::write_all(),
not sure if it's worth keeping our wrapper
There was a problem hiding this comment.
As far as I'm aware, write_all does not work with raw file descriptors, otherwise I would have used it. Since we need to write to stdout and stderr, we can't use a File in all cases.
a713e80 to
e574d6c
Compare
There is no need to use `'\0'`, we can use `0u8` instead. `maxaccum` does not clearly indicate what it represents; use `accum_capacity` instead. To get the size of `accum`, we can use `accum.len()`, which is easier to understand. Use `copy_from_slice` instead of `std::ptr::copy`. This avoids the unsafe block, at the cost of a runtime length check. If we don't want this change for performance reasons, we might consider using `std::ptr::copy_nonoverlapping` here (which is done by `copy_from_slice` internally). Part of #12008
7c73cf0 to
d1a2193
Compare
Interrupts should not prevent output from being completed, unless they result in control flow not continuing where it was interrupted. Return results containing errors, so callers have more options for error handling. Returning the number of bytes written on success is no longer necessary, since success now means that all input bytes were written. In `src/io.rs` some changes are needed since `EINTR` is no longer a possible return value. I also removed the comments on the old code and the `sigcheck` code. Not sure if this should be kept.
d1a2193 to
5d89e40
Compare
|
The Fluent PR no longer depends on this, so there is no immediate need to get these changes merged. If you'd like, we can just close this PR. I don't think I'll look much more into this any time soon. |
|
The Fluent PR no longer depends on this, so there is no immediate
need to get these changes merged. If you'd like, we can just close
this PR. I don't think I'll look much more into this any time soon.
there's different opinions on how to manage the PR/issue list.
I don't really care either way (since I can additionally keep my private todo-list)
I'd keep it open because at least the bug (?) about fish_indent not
being cancelable (which is not reported elsewhere yet AFAIK) should
have an obvious fix, and then we can figure out if the other things.
|
Interrupts should not prevent output from being completed, unless they
result in control flow not continuing where it was interrupted.
Return results containing errors, so callers have more options for error
handling. Returning the number of bytes written on success is no longer
necessary, since success now means that all input bytes were written.
In
src/io.rssome changes are needed sinceEINTRis no longer apossible return value. I also removed the comments on the old code and
the
sigcheckcode. Not sure if this should be kept.