-
Notifications
You must be signed in to change notification settings - Fork 1.3k
functools.partial #5825
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
functools.partial #5825
Conversation
WalkthroughA full implementation of the Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant PyPartial
participant Callable
participant VM
User->>PyPartial: Create partial(callable, *args, **kwargs)
PyPartial->>PyPartial: Flatten nested partials, merge args/kwargs
User->>PyPartial: Call partial(*call_args, **call_kwargs)
PyPartial->>PyPartial: Merge stored and call-time args/kwargs
PyPartial->>Callable: Call underlying callable with merged args/kwargs
Callable-->>User: Return result
Poem
📜 Recent review detailsConfiguration used: CodeRabbit UI 📒 Files selected for processing (1)
🧰 Additional context used📓 Path-based instructions (1)`**/*.rs`: Follow the default rustfmt code style; use 'cargo fmt' to format Rust code. Always run clippy to lint Rust code ('cargo clippy') before completing tasks, and fix any war...
⏰ Context from checks skipped due to timeout of 90000ms (11)
🔇 Additional comments (4)
✨ Finishing Touches
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 0
🧹 Nitpick comments (1)
vm/src/stdlib/functools.rs (1)
247-247
: Consider iterating directly over the keywords without cloning.Cloning the entire dictionary here is unnecessary since you're already holding a read lock. You can iterate directly over the reference to improve performance.
Apply this diff to avoid the unnecessary clone:
- for (key, value) in inner.keywords.clone() { + for (key, value) in &*inner.keywords {
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
Lib/test/test_functools.py
(0 hunks)vm/src/stdlib/functools.rs
(2 hunks)
💤 Files with no reviewable changes (1)
- Lib/test/test_functools.py
⏰ Context from checks skipped due to timeout of 90000ms (11)
- GitHub Check: Check Rust code with rustfmt and clippy
- GitHub Check: Check the WASM package and demo
- GitHub Check: Run snippets and cpython tests on wasm-wasi
- GitHub Check: Run tests under miri
- GitHub Check: Run rust tests (windows-latest)
- GitHub Check: Run snippets and cpython tests (macos-latest)
- GitHub Check: Run rust tests (macos-latest)
- GitHub Check: Run rust tests (ubuntu-latest)
- GitHub Check: Run snippets and cpython tests (ubuntu-latest)
- GitHub Check: Run snippets and cpython tests (windows-latest)
- GitHub Check: Ensure compilation on various targets
🔇 Additional comments (2)
vm/src/stdlib/functools.rs (2)
265-323
: Well-implemented repr with proper recursive handling and module qualification.The repr implementation correctly handles all edge cases including recursive repr, proper formatting of keyword arguments, and appropriate module qualification for different contexts.
45-323
: Excellent implementation of functools.partial with comprehensive feature coverage.This implementation successfully provides:
- Thread-safe access to partial state using PyRwLock
- Correct handling of nested partials through flattening
- Proper pickling support with reduce and setstate
- Robust error handling with descriptive messages
- Accurate repr formatting that matches Python's behavior
The removal of the
@unittest.expectedFailure
decorator in the test file confirms this implementation now passes all tests.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 1
🧹 Nitpick comments (3)
vm/src/stdlib/functools.rs (3)
103-138
: State restoration has robust validation but consider simplifying tuple handling.The validation logic is thorough, checking tuple length, callable validity, and type correctness. However, the tuple conversion logic for handling subclasses could be simplified.
Consider using a helper function for tuple conversion to reduce code duplication:
+ fn ensure_base_tuple(obj: &PyObjectRef, vm: &VirtualMachine) -> PyResult<PyRef<PyTuple>> { + match obj.clone().downcast::<PyTuple>() { + Ok(tuple) if tuple.class().is(vm.ctx.types.tuple_type) => Ok(tuple), + Ok(_) | Err(_) => { + let elements: Vec<PyObjectRef> = obj.try_to_value(vm)?; + Ok(vm.ctx.new_tuple(elements)) + } + } + } + - // Always convert to base tuple, even if it's a subclass - let args_tuple = match args.clone().downcast::<PyTuple>() { - Ok(tuple) if tuple.class().is(vm.ctx.types.tuple_type) => tuple, - _ => { - // It's a tuple subclass, convert to base tuple - let elements: Vec<PyObjectRef> = args.try_to_value(vm)?; - vm.ctx.new_tuple(elements) - } - }; + let args_tuple = ensure_base_tuple(args, vm)?;
265-323
: Representation implementation is comprehensive but could be optimized.The representation logic handles recursion correctly and provides appropriate formatting. However, the qualified name logic could be simplified and the string handling could be more efficient.
Consider simplifying the qualified name logic and using more efficient string operations:
// Check if this is a subclass by comparing with the base partial type - let is_subclass = !zelf.class().is(PyPartial::class(&vm.ctx)); - - let qualified_name = if !is_subclass { + let qualified_name = if zelf.class().is(PyPartial::class(&vm.ctx)) { // For the base partial class, always use functools.partial "functools.partial".to_string() } else { // For subclasses, check if they're defined in __main__ or test modules match module.downcast::<crate::builtins::PyStr>() { Ok(module_str) => { let module_name = module_str.as_str(); match module_name { - "builtins" | "" | "__main__" => class_name.to_string(), + "builtins" | "" | "__main__" => class_name.to_owned(), name if name.starts_with("test.") || name == "test" => { - // For test modules, just use the class name without module prefix - class_name.to_string() + class_name.to_owned() } _ => format!("{}.{}", module_name, class_name), } } - Err(_) => class_name.to_string(), + Err(_) => class_name.to_owned(), } };
275-289
: Argument formatting logic could be more robust.The representation of arguments and keywords is generally correct, but the string conversion for non-string keys could be handled more defensively.
Consider adding error handling for the string conversion:
for (key, value) in inner.keywords.clone() { // For string keys, use them directly without quotes let key_part = if let Ok(s) = key.clone().downcast::<crate::builtins::PyStr>() { s.as_str().to_owned() } else { // For non-string keys, convert to string using __str__ - key.str(vm)?.as_str().to_owned() + match key.str(vm) { + Ok(s) => s.as_str().to_owned(), + Err(_) => "<unprintable key>".to_owned(), + } }; - let value_str = value.repr(vm)?; + let value_str = value.repr(vm).unwrap_or_else(|_| vm.ctx.new_str("<unprintable>")); parts.push(format!("{}={}", key_part, value_str.as_str())); }This prevents representation failures from propagating and provides more user-friendly output.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (3)
Lib/test/test_functools.py
(0 hunks)Lib/test/test_threading.py
(0 hunks)vm/src/stdlib/functools.rs
(2 hunks)
💤 Files with no reviewable changes (2)
- Lib/test/test_threading.py
- Lib/test/test_functools.py
🧰 Additional context used
📓 Path-based instructions (1)
`**/*.rs`: Follow the default rustfmt code style; use 'cargo fmt' to format Rust code. Always run clippy to lint Rust code ('cargo clippy') before completing tasks, and fix any war...
**/*.rs
: Follow the default rustfmt code style; use 'cargo fmt' to format Rust code.
Always run clippy to lint Rust code ('cargo clippy') before completing tasks, and fix any warnings or lints introduced by your changes.
vm/src/stdlib/functools.rs
⏰ Context from checks skipped due to timeout of 90000ms (11)
- GitHub Check: Run snippets and cpython tests on wasm-wasi
- GitHub Check: Run tests under miri
- GitHub Check: Check the WASM package and demo
- GitHub Check: Run snippets and cpython tests (ubuntu-latest)
- GitHub Check: Run snippets and cpython tests (macos-latest)
- GitHub Check: Run rust tests (windows-latest)
- GitHub Check: Check Rust code with rustfmt and clippy
- GitHub Check: Run snippets and cpython tests (windows-latest)
- GitHub Check: Run rust tests (ubuntu-latest)
- GitHub Check: Run rust tests (macos-latest)
- GitHub Check: Ensure compilation on various targets
🔇 Additional comments (6)
vm/src/stdlib/functools.rs (6)
5-16
: Imports look comprehensive and well-organized.The imports cover all necessary components for the implementation including thread safety (
PyRwLock
), function handling (FuncArgs
,KwArgs
), and various traits. Good use of theindexmap
crate for maintaining key order in keyword arguments.
45-58
: Struct design follows Rust best practices.The use of
PyRwLock<PyPartialInner>
provides thread safety while allowing both read and write access. The inner struct cleanly separates the mutable state with appropriate types (PyObjectRef
,PyRef<PyTuple>
,PyRef<PyDict>
).
61-74
: Property getters are correctly implemented.The getters properly acquire read locks and clone the references, which is safe and follows Rust conventions. The property names match Python's
functools.partial
interface.
76-101
: Pickling implementation follows Python semantics correctly.The
__reduce__
method properly constructs the expected tuple format(type, (func,), state)
where state contains(func, args, keywords, __dict__)
. The handling of the instance dictionary (__dict__
) correctly checks for existence and emptiness before including it.
139-192
: Dictionary handling and instance state restoration are well-implemented.The logic correctly handles None values, dictionary subclasses, and instance dictionary updates. The approach of clearing and rebuilding the dictionary ensures clean state restoration.
195-233
: Constructor correctly handles nested partial flattening.The implementation properly validates the first argument is callable and correctly flattens nested partial objects by merging their arguments and keywords. The keyword handling uses
intern_str
which is appropriate for optimization.
vm/src/stdlib/functools.rs
Outdated
impl Callable for PyPartial { | ||
type Args = FuncArgs; | ||
|
||
fn call(zelf: &Py<Self>, args: FuncArgs, vm: &VirtualMachine) -> PyResult { | ||
let inner = zelf.inner.read(); | ||
let mut combined_args = inner.args.as_slice().to_vec(); | ||
combined_args.extend_from_slice(&args.args); | ||
|
||
// Merge keywords from self.keywords and args.kwargs | ||
let mut final_kwargs = IndexMap::new(); | ||
|
||
// Add keywords from self.keywords | ||
for (key, value) in inner.keywords.clone() { | ||
let key_str = key | ||
.downcast::<crate::builtins::PyStr>() | ||
.map_err(|_| vm.new_type_error("keywords must be strings".to_owned()))?; | ||
final_kwargs.insert(key_str.as_str().to_owned(), value); | ||
} | ||
|
||
// Add keywords from args.kwargs (these override self.keywords) | ||
for (key, value) in args.kwargs { | ||
final_kwargs.insert(key, value); | ||
} | ||
|
||
inner | ||
.func | ||
.call(FuncArgs::new(combined_args, KwArgs::new(final_kwargs)), vm) | ||
} | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Callable implementation has a potential issue with keyword validation.
The callable logic correctly merges arguments and keywords, but there's an inconsistency in keyword validation that could cause runtime errors.
The keyword validation in the callable implementation differs from the constructor. In the constructor, all keywords are converted to interned strings, but in the callable, existing keywords are validated to be strings and could fail:
// Add keywords from self.keywords
for (key, value) in inner.keywords.clone() {
- let key_str = key
- .downcast::<crate::builtins::PyStr>()
- .map_err(|_| vm.new_type_error("keywords must be strings".to_owned()))?;
- final_kwargs.insert(key_str.as_str().to_owned(), value);
+ // Keywords should already be strings from construction, but handle gracefully
+ let key_str = if let Ok(s) = key.downcast::<crate::builtins::PyStr>() {
+ s.as_str().to_owned()
+ } else {
+ // This shouldn't happen if construction is correct, but handle it
+ key.str(vm)?.as_str().to_owned()
+ };
+ final_kwargs.insert(key_str, value);
This ensures consistency with the construction logic where keywords are always stored as strings.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
impl Callable for PyPartial { | |
type Args = FuncArgs; | |
fn call(zelf: &Py<Self>, args: FuncArgs, vm: &VirtualMachine) -> PyResult { | |
let inner = zelf.inner.read(); | |
let mut combined_args = inner.args.as_slice().to_vec(); | |
combined_args.extend_from_slice(&args.args); | |
// Merge keywords from self.keywords and args.kwargs | |
let mut final_kwargs = IndexMap::new(); | |
// Add keywords from self.keywords | |
for (key, value) in inner.keywords.clone() { | |
let key_str = key | |
.downcast::<crate::builtins::PyStr>() | |
.map_err(|_| vm.new_type_error("keywords must be strings".to_owned()))?; | |
final_kwargs.insert(key_str.as_str().to_owned(), value); | |
} | |
// Add keywords from args.kwargs (these override self.keywords) | |
for (key, value) in args.kwargs { | |
final_kwargs.insert(key, value); | |
} | |
inner | |
.func | |
.call(FuncArgs::new(combined_args, KwArgs::new(final_kwargs)), vm) | |
} | |
} | |
impl Callable for PyPartial { | |
type Args = FuncArgs; | |
fn call(zelf: &Py<Self>, args: FuncArgs, vm: &VirtualMachine) -> PyResult { | |
let inner = zelf.inner.read(); | |
let mut combined_args = inner.args.as_slice().to_vec(); | |
combined_args.extend_from_slice(&args.args); | |
// Merge keywords from self.keywords and args.kwargs | |
let mut final_kwargs = IndexMap::new(); | |
// Add keywords from self.keywords | |
for (key, value) in inner.keywords.clone() { | |
// Keywords should already be strings from construction, but handle gracefully | |
let key_str = if let Ok(s) = key.downcast::<crate::builtins::PyStr>() { | |
s.as_str().to_owned() | |
} else { | |
// This shouldn't happen if construction is correct, but handle it | |
key.str(vm)?.as_str().to_owned() | |
}; | |
final_kwargs.insert(key_str, value); | |
} | |
// Add keywords from args.kwargs (these override self.keywords) | |
for (key, value) in args.kwargs { | |
final_kwargs.insert(key, value); | |
} | |
inner | |
.func | |
.call(FuncArgs::new(combined_args, KwArgs::new(final_kwargs)), vm) | |
} | |
} |
🤖 Prompt for AI Agents
In vm/src/stdlib/functools.rs around lines 235 to 263, the call method in the
Callable implementation for PyPartial inconsistently validates keywords by
downcasting keys to PyStr and returning an error if they are not strings, unlike
the constructor which converts all keywords to interned strings. To fix this,
ensure that the callable method processes keywords the same way as the
constructor by converting all keyword keys to interned strings before merging,
maintaining consistency and preventing runtime errors.
Summary by CodeRabbit
New Features
partial
callable wrapper, enabling creation, calling, pickling, state restoration, and detailed string representation of partial objects.Bug Fixes