can be factored out into a helper function, and lets DCHECK as well, as I do not see why should we allow callers to call this function with no data

• I'll put a helper, thanks - see my other comment about iovec_utils.cc/h. I'm going to add part of it there.
• I'll add a DCHECK since I can assume we never want to send 0 data in debug.
• Production: Standard POSIX behaviour (specifically writev and sendmsg) dictates that if you pass a valid iovec array where the sum of lengths is zero, the system call simply returns 0 and does nothing. It is not an error, and we cannot crash on it.

should we DCHECK here that engine_->OutputPending() == 0 ?

I don't think so:

• TrySend does not control SSL engine exclusively. I have no idea who called the engine before, maybe via other function. Maybe oms read on the socket generated SSL metadata on the output , and other cases.
• TrySend can run concurrently with some Read operation, so again OutputPending() can become non-zero.

I am not sure it's the best approach.

1. batching writes is beneficial - we do not want to send a packet per few bytes, we already had such bugs.
2. if engine_->OutputPending() > 0 that should mean that there is an asynchronous process that takes care of it, imho.

1. Regarding Batching: You are right that the current "Flush-Push-Flush" loop could result in small packets (one per iovec), which isn't ideal for performance. I will try to refactor this to Send only once in multiple PushToEngine calls before triggering a flush to the upstream socket. If this makes the non-blocking logic too complex for this single PR, I will merge this correct version first and optimize this in a dedicated follow-up PR to keep the changes manageable.

2. Regarding OutputPending & Async Flushing: I have to disagree on the "asynchronous process" point. At the very top of TrySend, I check:
   if ((state_ & WRITE_IN_PROGRESS) != 0) return ... try_again;
   If there were any asynchronous process or background fiber currently flushing this buffer, WRITE_IN_PROGRESS would be set, and I would have exited immediately.
   Since I reached this line, I am the exclusive writer. There is no other active process responsible for this data. If I don't flush OutputPending here, the data will sit in the SSL BIO indefinitely (causing latency) until the next API call happens to trigger a write. Therefore, it is my responsibility to flush it. Also, this is used as a single place to flush between iteration (only when needed, in the optimised-to-be-written version).

1. Bug. what ensures that the write will be flushed when next_sock_ becomes available?
2. make_error_code(errc::resource_unavailable_try_again); is wrong as you already consumed some of the input data and it was copied to ssl engine. From a caller perspective - they need to retry the entire operation, so they will try to push the same data again

I believe that for partial writes code is correct, there is only one edge case I will mention at the end which is a bit "tricky" (full write with partial flash of the SSL engine output).

====================

• "what will push the data eventually(1)" ?
  Since I return total_bytes_sent > 0, the caller MUST treats this as a successful partial write. Standard non-blocking socket behaviour implies the caller will eventually call TrySend (or TryRecv) again. iT MUST call TrySend again since the partial write implied also "try again after advancing your iovec".

When user do call TrySend again (even with new data), the very first block of this function:
if (engine_->OutputPending() > 0) { ... }
ensures that we attempt to flush the pending ciphertext before accepting any new user data.
Regarding the error handling (swallowing EAGAIN/errors): this mimics standard POSIX write() semantics. If a write partially succeeds but then hits a network error or blocks, we report the success first (we return total_bytes_sent>0) . The error remains pending and will be returned on the next call (when total_bytes_sent is 0). If error is fatal, it will still "wait for" next user call, if it's non-fatal - the next TrySend/TryRecv might not encounter the error anymore. There is one edge case to discuss at the end of this comment.

=========================
Regarding the duplication concern (2):
I believe the duplication concern does not exist, and is resolved by the return logic at the end of the function.
You are correct that returned_status is set to try_again, but notice the check at the very end:

if (total_bytes_sent > 0) { 
     return total_bytes_sent;
}

Even if returned_status contains an error (like EAGAIN or a socket error), if I have successfully pushed any bytes to the engine (total_bytes_sent > 0), I return that positive count, effectively masking the error for this specific call. This ensures that:

• The caller sees a positive return value (partial write).
• The caller advances their buffer pointers by total_bytes_sent.
• The caller invokes TrySend again with the remaining (new) data.

Therefore, no data is duplicated. The user never retries with the same data because they received a positive confirmation for the chunk that was processed.

====================
Regarding the "Full Write, Partial Flush" scenario (this one is tricky):

Consider the case where the user sends 1000 bytes. We successfully push all 1000 bytes into the SSL engine (encrypting them), but the upstream socket only accepts 500 bytes of the ciphertext before returning EAGAIN.
In this state:
I must return 1000 (success). All the user data has been consumed and encrypted (but some of it not sent yet on the upstream socket). If I return EAGAIN here, the user will retry sending the same 1000 bytes. This would encrypt the data a second time, corrupting the TLS stream (duplication).

So, what ensures the flush?

My claim is that TrySend is a "best effort" function with minimalist return value which cant reflect all complex situations and must return the number of bytes sent even if there was an error. It's the duty of the caller to make sure the TLS engine output buffer is flashed by calling again to TrySend/TryRecv or using AsyncReq.

In details:

1. Immediate Retry: Since I return a positive byte count, the non-blocking contract implies the user (or upper layer) will continue to call TrySend (to send more data) or TryRecv (to wait for a response). Both functions begin by attempting to flush the engine buffer. But what if user do not call again since they do not want to send or receive more data? for that we have mechanism 2.
2. Async Layer Safety: For the async/fiber implementation (AsyncReq), we explicitly handle this state. In AsyncReq::MaybeSendOutputAsyncWithRead (and AsyncReadSome), we check engine_->OutputPending() and call StartUpstreamWrite() if positive This one register for write on the uostream socker in next_socket->AsyncWriteSome. If there is buffered ciphertext, we register in that function (next_socket_->AsyncWriteSome) for a Write event (EPOLLOUT) , even if the user requested a Read, preventing any deadlock.

we concluded that function is correct for all cases except one: when all user data sent and there is pending data n engine output buffer. In that case we would like to make this function "send and forget". Code will start an async process to make sure engine output buffer is flushed into upstream socket.

2. same thing - what will push the data eventually?

• Regarding a general error case (General Error): This follows standard POSIX partial write semantics. If we successfully processed some bytes but then hit a hard error (e.g., Broken Pipe), we return the total_bytes_sent first . The next call will attempt to flush, hit the error immediately (with total_bytes_sent == 0), and correctly return the error to the user. The edge case of full write will partial flash - see here at the end: feat(tls): implement non-blocking TrySend with async flush offload #516 (comment)

we concluded that function is correct for all cases except one: when all user data sent and there is pending data n engine output buffer. In that case we would like to make this function "send and forget". Code will start an async process to make sure engine output buffer is flushed into upstream socket.