feat: SFTP connection pool #3198
Conversation
There was a problem hiding this comment.
Pull Request Overview
This PR implements SFTP connection pooling for the Yazi file manager, enabling support for remote file system operations over SFTP. The changes introduce a complete SFTP provider with authentication methods (password, key file, and SSH agent), connection management with pooling for efficiency, and integration with the existing provider architecture.
- Adds comprehensive SFTP connection management with multiple authentication methods
- Implements connection pooling with automatic reconnection on failure
- Refactors the provider system to support remote file operations through a unified interface
Reviewed Changes
Copilot reviewed 30 out of 31 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| yazi-vfs/src/config/vfs.rs | Changes visibility of configuration methods to support SFTP provider configuration |
| yazi-vfs/src/config/provider.rs | Adds SFTP provider configuration with authentication fields and validation |
| yazi-shared/src/url/encode.rs | Exposes domain encoding for SFTP URL handling |
| yazi-shared/src/loc/loc.rs | Exposes byte access method for URL processing |
| yazi-shared/src/env.rs | Updates log level formatting to include module prefix |
| yazi-sftp/src/session.rs | Improves SFTP session management with better error handling and connection state tracking |
| yazi-sftp/src/requests/*.rs | Updates SFTP request structures to use borrowed references for better performance |
| yazi-sftp/src/packet.rs | Adds support for additional SFTP packet types |
| yazi-sftp/src/operator.rs | Updates SFTP operator API to use borrowed references |
| yazi-sftp/src/fs/*.rs | Updates file system structures and operations for better API design |
| yazi-fs/src/provider/sftp/sftp.rs | Implements complete SFTP provider with connection pooling and authentication |
| yazi-fs/src/provider/*.rs | Refactors provider system to support remote operations and unified file handling |
| yazi-fs/src/cwd.rs | Updates current working directory handling for remote file systems |
| yazi-boot/src/boot.rs | Updates boot process to handle absolute URL resolution |
| Cargo.toml | Updates dependency versions |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
| callback: Mutex<HashMap<u32, oneshot::Sender<Packet<'static>>>>, | ||
| pub(super) extensions: Mutex<HashMap<String, String>>, | ||
| } | ||
|
|
There was a problem hiding this comment.
The Drop implementation sends an empty vector to signal shutdown, but this behavior is not documented. Consider adding a comment explaining that an empty vector signals the session to close.
| // When the Session is dropped, send an empty vector through the channel to signal shutdown. | |
| // The receiving end interprets an empty vector as a request to close the session/channel. |
|
|
||
| let pool = *super::CONN.lock().entry(self.config).or_insert_with(|| { | ||
| Box::leak(Box::new(deadpool::managed::Pool::builder(self).build().unwrap())) | ||
| Box::leak(Box::new(deadpool::managed::Pool::builder(self).max_size(5).build().unwrap())) |
There was a problem hiding this comment.
The hardcoded max_size of 5 connections should be configurable. Consider adding this as a configuration option in the SFTP provider config.
| Box::leak(Box::new(deadpool::managed::Pool::builder(self).max_size(5).build().unwrap())) | |
| let max_size = self.config.max_connections.unwrap_or(5); | |
| Box::leak(Box::new(deadpool::managed::Pool::builder(self).max_size(max_size).build().unwrap())) |
| keepalive_interval: Some(std::time::Duration::from_secs(10)), | ||
| nodelay: true, |
There was a problem hiding this comment.
The hardcoded timeout values (30s inactivity, 10s keepalive) should be configurable. Consider adding these as optional fields to the ProviderSftp configuration.
| keepalive_interval: Some(std::time::Duration::from_secs(10)), | |
| nodelay: true, | |
| inactivity_timeout: self.config.inactivity_timeout.map(std::time::Duration::from_secs).or(Some(std::time::Duration::from_secs(30))), | |
| keepalive_interval: self.config.keepalive_interval.map(std::time::Duration::from_secs).or(Some(std::time::Duration::from_secs(10))), |
| let ft = ok_or_not_found!(child.file_type().await, continue); | ||
| let result = if ft.is_dir() { | ||
| remove_dir_all_impl(me, &child.path()).await | ||
| Box::pin(remove_dir_all_impl(me, &child.path())).await |
There was a problem hiding this comment.
Boxing the recursive call in remove_dir_all_impl could cause stack overflow for deeply nested directories. Consider using an iterative approach with a stack-based implementation instead of recursive boxing.
| // FIXME: set default: $SSH_AUTH_SOCK | ||
| pub identity_agent: Option<PathBuf>, |
There was a problem hiding this comment.
The FIXME comment indicates unimplemented functionality for SSH_AUTH_SOCK default. Either implement this feature or document why it's deferred.
Prepare for #611