Nats timeout #5528
Nats timeout #5528
Conversation
Signed-off-by: kfollesdal <[email protected]>
Signed-off-by: kfollesdal <[email protected]>
Signed-off-by: kfollesdal <[email protected]>
…ection error Signed-off-by: kfollesdal <[email protected]>
…reation Signed-off-by: kfollesdal <[email protected]>
Signed-off-by: kfollesdal <[email protected]>
There was a problem hiding this comment.
Pull request overview
Adds configurable NATS timeouts and improves initialization failures by validating JetStream stream existence early, with clearer error context and new tests.
Changes:
- Added
connection_timeout_secsandrequest_timeout_secsto NATS connect options (default 10s). - Added early stream existence verification during NATS input initialization and improved error context.
- Added tests covering connection errors, stream-not-found, and timeout behavior.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| docs.feldera.com/docs/connectors/sources/nats.md | Documents new NATS timeout configuration options. |
| crates/feldera-types/src/transport/nats.rs | Adds new config fields and defaults for NATS connection/request timeouts. |
| crates/adapters/src/transport/nats/input/config_utils.rs | Applies the new timeout options to async_nats::ConnectOptions. |
| crates/adapters/src/transport/nats/input.rs | Validates stream existence up-front and improves connection/consumer error context. |
| crates/adapters/src/transport/nats/input/test.rs | Adds tests for connection refused, stream-not-found, and timeout behavior. |
| Relation::empty(), | ||
| ); | ||
|
|
||
| assert_nats_connect_error(result, nonexistent_url, "Connection refused"); |
There was a problem hiding this comment.
The test asserts on the OS-/library-specific error string "Connection refused", which can vary in casing/wording across platforms and dependency versions. To make this test robust in CI, prefer asserting on an underlying std::io::ErrorKind::ConnectionRefused by walking the anyhow error chain (downcasting), or loosen the string match (e.g., case-insensitive contains of "refused").
| elapsed | ||
| ); | ||
|
|
||
| assert_nats_connect_error(result, non_routable_url, "timed out"); |
There was a problem hiding this comment.
This timeout test is likely flaky: connecting to 10.255.255.1 may fail with different errors (e.g., "Network is unreachable") or behave differently depending on CI networking, and error text "timed out" is not stable. A more deterministic approach is to run a local dummy TCP listener that accepts connections but never speaks NATS (so the client waits for the handshake until connection_timeout_secs), and/or assert on std::io::ErrorKind::TimedOut via error-chain downcasting rather than matching a string.
| assert_nats_connect_error(result, non_routable_url, "timed out"); | |
| // Ensure we got a connection error and that it references the configured URL, | |
| // without relying on OS-specific timeout wording. | |
| let err = result.expect_err("Expected NATS connection to fail"); | |
| let err_str = format!("{err:?}"); | |
| assert!( | |
| err_str.contains(&non_routable_url), | |
| "Connection error should mention server URL {non_routable_url}, got: {err_str}" | |
| ); |
| "Failed to create consumer on stream '{}' (start_sequence={}, deliver_policy={:?}, filter_subjects={:?})", | ||
| stream_name, |
There was a problem hiding this comment.
The updated error context for consumer creation no longer includes the consumer name (configured name and/or the generated unique name), which can make debugging harder when multiple consumers are involved. Consider including consumer_config.name (or the final created name if available at this call site) in this message in addition to the other parameters.
| "Failed to create consumer on stream '{}' (start_sequence={}, deliver_policy={:?}, filter_subjects={:?})", | |
| stream_name, | |
| "Failed to create consumer on stream '{}' (consumer_name={:?}, start_sequence={}, deliver_policy={:?}, filter_subjects={:?})", | |
| stream_name, | |
| consumer_config.name, |
| #[serde(default, skip_serializing_if = "is_default")] | ||
| pub auth: Auth, | ||
|
|
||
| /// Connection timeout |
There was a problem hiding this comment.
The field name is connection_timeout_secs, but the doc comment header doesn’t mention the unit. Consider changing it to “Connection timeout in seconds.” (similar to the request_timeout_secs comment) for consistency and clarity in generated docs/schema.
| /// Connection timeout | |
| /// Connection timeout in seconds. |
Signed-off-by: feldera-bot <[email protected]>
This is a copy of #5510, by @kfollesdal, into the main repo to allow it to go through CI (our CI system is unfortunately not yet capable of handling out-of-repo branches).
Checklist