Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@iskng
Copy link
Contributor

@iskng iskng commented Oct 6, 2025

Overview

  • Introduces an additive extension point to run the cluster protocol over any reliable, ordered,
    bidirectional byte stream.
  • Keeps current TCP/TLS defaults and APIs intact; no wire or behavioral changes to existing transports.
  • Adds documentation and integration tests (in‑memory and Unix sockets) to demonstrate the new path.

Why

  • Decouple transport from protocol/actors so downstreams can use QUIC, WebSocket, SSH tunnels, Unix
    domain sockets, named pipes, or in‑memory streams.
  • Improve testability with in‑memory streams without sockets.
  • Keep the change minimal and upstream‑friendly: no new dependencies and unchanged defaults.

What Changed

  • Transport seam:
    • New ClusterBidiStream trait with split() -> (BoxRead, BoxWrite) and optional peer_label/
      local_label.
    • New aliases BoxRead/BoxWrite for boxed tokio::io halves.
    • New NodeServerMessage::ConnectionOpenedExternal { stream, is_server }.
    • New NetworkStream::External { reader, writer, peer_label, local_label }.
  • Session IO:
    • net::Session extended to read/write the External halves alongside existing Raw/TLS branches.
    • Framing remains the same (length prefix + prost payload).
  • Client helper:
    • client_connect_external(&node_server, Box).await.
  • Diagnostics:
    • Manual Debug for NetworkStream to handle External (prints labels, not halves).
  • Docs:
    • README “Bring Your Own Transport” section with a concise example.
  • Tests (no new crates):
    • In‑memory (DuplexStream): external_transport_auth_and_ready
    • Unix Domain Sockets (Unix only): external_unix_auth_and_ready

Backwards Compatibility

  • TCP/TLS connect/listen unchanged.
  • On-the-wire format unchanged.
  • Public API is additive; no removed or renamed items.
  • NetworkStream now has a manual Debug impl.

API Summary

  • New trait: ractor_cluster::ClusterBidiStream
  • New types: ractor_cluster::{BoxRead, BoxWrite}
  • New message: NodeServerMessage::ConnectionOpenedExternal { stream: Box,
    is_server: bool }
  • New helper: ractor_cluster::client_connect_external(...)

Usage

  • Implement ClusterBidiStream for your connected stream type and inject:
    • node_server.cast(NodeServerMessage::ConnectionOpenedExternal { stream: Box::new(MyStream),
      is_server: true/false })?;
    • Or: ractor_cluster::client_connect_external(&node_server, Box::new(MyStream)).await?;
  • Framing/auth/control/RemoteActor/PG sync remain centralized and unchanged.

Tests

  • Build: cargo check
  • Integration tests:
    • All: cargo test -p ractor_cluster_integration_tests
    • Duplex: cargo test -p ractor_cluster_integration_tests -- tests::external_transport
    • UDS (Unix): cargo test -p ractor_cluster_integration_tests -- tests::external_unix

Implementation Notes

  • NetworkStream::peer_addr()/local_addr() return 0.0.0.0:0 for External; NodeServer uses labels for
    NodeServerSessionInformation.peer_addr.
  • Raw TCP retains .readable()/.writable() readiness; TLS and External use straightforward async reads/
    writes.
  • No new dependencies added to ractor_cluster; the test crate only enables additional Tokio features
    (net, io-util).

Follow‑ups (Out of Scope Here)

  • Optional acceptor/dialer traits for generic listeners/clients.
  • Standardize writer length prefix to u64 (reader already reads u64); left unchanged to avoid any wire
    delta.

Checklist

  • Additive API only; no breaking changes
  • Docs updated (ractor_cluster/README.md)
  • Manual Debug for NetworkStream
  • Integration tests for external transports (Duplex, UDS)
  • No new dependencies in ractor_cluster

Touched Files (high level)

  • ractor_cluster/src/net.rs: trait/types, External variant, manual Debug
  • ractor_cluster/src/net/session.rs: External halves support
  • ractor_cluster/src/node.rs: ConnectionOpenedExternal handler
  • ractor_cluster/src/node/client.rs: client_connect_external
  • ractor_cluster/src/lib.rs: re-exports
  • ractor_cluster/README.md: BYO Transport docs
  • ractor_cluster_integration_tests/*: new tests and minimal tokio feature enablement

@codecov
Copy link

codecov bot commented Oct 7, 2025

Codecov Report

❌ Patch coverage is 43.00000% with 57 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.82%. Comparing base (954f20b) to head (0e54999).

Files with missing lines Patch % Lines
ractor_cluster/src/net.rs 6.66% 42 Missing ⚠️
ractor_cluster/src/node/client.rs 0.00% 11 Missing ⚠️
ractor_cluster/src/node.rs 88.88% 4 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #391      +/-   ##
==========================================
+ Coverage   82.48%   85.82%   +3.34%     
==========================================
  Files          71       71              
  Lines       12918    13018     +100     
==========================================
+ Hits        10655    11173     +518     
+ Misses       2263     1845     -418     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Owner

@slawlor slawlor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is super cool! Thanks for the addition!

Any chance we could add a e2e test on the docker playground to go with it? I'm not sure how much boiler plate is required to setup a custom stream, but it would help validate that a fully setup networked node works as expected.

@slawlor
Copy link
Owner

slawlor commented Oct 13, 2025

You'd need to create a new "env" for Docker to automate the test similar to this: https://github.com/slawlor/ractor/blob/main/ractor_cluster_integration_tests/envs/auth-handshake.env

Then add it to the test matrix here

https://github.com/slawlor/ractor/blob/main/.github/workflows/integration.yaml

* Add `ClusterBidiStream` trait for injecting custom network transports.
* Extend `NetworkStream` with an `External` variant for user-provided streams.
* Implement `NodeServerMessage::ConnectionOpenedExternal` to accept new
  transports.
* Provide `client_connect_external` helper for easy client-side integration.
* Update `README.md` with detailed instructions and examples for custom
  transports.
* Ensure full compatibility; existing TCP and TLS connections are unaffected.
@slawlor slawlor force-pushed the feature/external-transport branch from 8d4fa5b to d4ecbf8 Compare October 13, 2025 16:05
iskng and others added 4 commits October 14, 2025 18:00
- Implemented a comprehensive end-to-end integration test for the
  `ractor_cluster` external transport API.
- The test utilizes Docker containers to simulate a multi-node cluster,
  binding custom TCP listeners and dialling peers directly.
- Replaced previous in-memory duplex stream tests with real TCP
  connections for more realistic validation.
- Introduced `ExternalProbeActor` to verify remote messaging and RPC
  functionality over the custom transport.
- Integrated the new test into the `.github/workflows/integration.yaml`
  CI pipeline to ensure continuous validation.
- Refactor `wait_for_session_ready` to handle authenticated-only sessions
  that may not fully establish or may close post-probe.
- Introduce `SESSION_AUTH_GRACE_MS` to allow flexibility for authenticated
  but not-yet-ready sessions to complete their purpose.
- Add `wait_for_remote_probe_presence` for tests where the local node
  discovers remote actors without establishing a full session.
- Implement `wait_for_session_teardown` to verify proper session cleanup
  when the local node is configured to listen.
- Update `ExternalProbeActor` to explicitly signal probe completion,
  enabling more robust session state verification.
- Conditionally apply session and teardown waiting based on
  `config.listen_port` for diverse test scenarios.
… benches

* Disambiguated `JoinHandle` types using `RactorJoinHandle` and
  `TokioJoinHandle` aliases for improved clarity.
* Updated `spawn_external_listener` and `cleanup` signatures to use
  the explicit `JoinHandle` aliases in cluster integration tests.
* Removed redundant `let _ =` bindings during `async-std` benchmark
  runtime initialization.
* Replaced `|v| Some(v)` closures with direct `Some` function calls
  for `output_port.subscribe` in actor benchmarks.
* Marked `_since` as unused in `wait_for_session_ready` destructuring
  for clearer intent.
@iskng
Copy link
Contributor Author

iskng commented Oct 17, 2025

Sry for the spam, was trying to get it done remotely.

I could add other more interesting options like quic, what I'm using it for, but the boiler plate seemed a bit heavy for the moment.

Thanks for all your work on this, Ractor is Awesome!!

@slawlor slawlor merged commit be3b23e into slawlor:main Oct 17, 2025
23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants