Thanks to visit codestin.com
Credit goes to github.com

Skip to content

functools.partial #5825

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jun 23, 2025
Merged

functools.partial #5825

merged 2 commits into from
Jun 23, 2025

Conversation

youknowone
Copy link
Member

@youknowone youknowone commented Jun 23, 2025

Summary by CodeRabbit

  • New Features

    • Introduced comprehensive support for the partial callable wrapper, enabling creation, calling, pickling, state restoration, and detailed string representation of partial objects.
  • Bug Fixes

    • Updated tests to reflect that attribute immutability for partial objects now passes as expected.
    • Marked the atexit-related test as passing, indicating improved reliability in exit handling.

Copy link

coderabbitai bot commented Jun 23, 2025

Walkthrough

A full implementation of the partial class is added to the _functools module in Rust, closely following Python's functools.partial. This includes construction, argument merging, pickling, state restoration, and representation. Corresponding test adjustments remove @unittest.expectedFailure decorators from tests in test_functools.py and test_threading.py, indicating these tests are now expected to pass.

Changes

File(s) Change Summary
Lib/test/test_functools.py, Lib/test/test_threading.py Removed @unittest.expectedFailure decorators and related TODO comments from tests now expected to pass.
vm/src/stdlib/functools.rs Added full Rust implementation of partial class, including construction, calling, pickling, state restoration, and representation.

Sequence Diagram(s)

sequenceDiagram
    participant User
    participant PyPartial
    participant Callable
    participant VM

    User->>PyPartial: Create partial(callable, *args, **kwargs)
    PyPartial->>PyPartial: Flatten nested partials, merge args/kwargs
    User->>PyPartial: Call partial(*call_args, **call_kwargs)
    PyPartial->>PyPartial: Merge stored and call-time args/kwargs
    PyPartial->>Callable: Call underlying callable with merged args/kwargs
    Callable-->>User: Return result
Loading

Poem

In the garden of functools, a partial appears,
Now written in Rust, it conquers old fears.
With merging of args and pickling in tow,
No longer expected to fail, the tests now all glow.
🥕 Cheers to new features, as code rabbits hop—
Our partial’s complete, from bottom to top!


📜 Recent review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between be05bc1 and caefb8e.

📒 Files selected for processing (1)
  • vm/src/stdlib/functools.rs (2 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
`**/*.rs`: Follow the default rustfmt code style; use 'cargo fmt' to format Rust code. Always run clippy to lint Rust code ('cargo clippy') before completing tasks, and fix any war...

**/*.rs: Follow the default rustfmt code style; use 'cargo fmt' to format Rust code.
Always run clippy to lint Rust code ('cargo clippy') before completing tasks, and fix any warnings or lints introduced by your changes.

  • vm/src/stdlib/functools.rs
⏰ Context from checks skipped due to timeout of 90000ms (11)
  • GitHub Check: Run snippets and cpython tests on wasm-wasi
  • GitHub Check: Check the WASM package and demo
  • GitHub Check: Run tests under miri
  • GitHub Check: Run rust tests (windows-latest)
  • GitHub Check: Run rust tests (ubuntu-latest)
  • GitHub Check: Run rust tests (macos-latest)
  • GitHub Check: Run snippets and cpython tests (macos-latest)
  • GitHub Check: Check Rust code with rustfmt and clippy
  • GitHub Check: Run snippets and cpython tests (windows-latest)
  • GitHub Check: Ensure compilation on various targets
  • GitHub Check: Run snippets and cpython tests (ubuntu-latest)
🔇 Additional comments (4)
vm/src/stdlib/functools.rs (4)

45-74: Well-designed struct with proper thread safety.

The use of PyRwLock for thread-safe access to the inner state is appropriate, and the property getters correctly use read locks. The separation of mutable state into PyPartialInner provides clean locking semantics.


76-192: Comprehensive pickling implementation with robust validation.

The __reduce__ and __setstate__ methods properly handle all aspects of pickling, including:

  • Correct pickle protocol format
  • Thorough validation of state tuple structure
  • Proper handling of tuple/dict subclasses by converting to base types
  • Correct instance dictionary management

195-233: Well-implemented constructor with smart nested partial handling.

The constructor correctly:

  • Validates input arguments
  • Unwraps nested partials to avoid unnecessary wrapping layers
  • Consistently stores keywords as interned strings
  • Properly merges arguments and keywords from nested partials

265-320: Excellent repr implementation with comprehensive module handling.

The representation correctly:

  • Prevents infinite recursion with ReprGuard
  • Formats output to match Python's partial representation
  • Handles module qualification intelligently, with special cases for test modules and __main__
  • Always uses "functools.partial" for the base class, maintaining compatibility
✨ Finishing Touches
  • 📝 Generate Docstrings

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Explain this complex logic.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai explain this code block.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and explain its main purpose.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (1)
vm/src/stdlib/functools.rs (1)

247-247: Consider iterating directly over the keywords without cloning.

Cloning the entire dictionary here is unnecessary since you're already holding a read lock. You can iterate directly over the reference to improve performance.

Apply this diff to avoid the unnecessary clone:

-            for (key, value) in inner.keywords.clone() {
+            for (key, value) in &*inner.keywords {
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 9952c97 and 960f87b.

📒 Files selected for processing (2)
  • Lib/test/test_functools.py (0 hunks)
  • vm/src/stdlib/functools.rs (2 hunks)
💤 Files with no reviewable changes (1)
  • Lib/test/test_functools.py
⏰ Context from checks skipped due to timeout of 90000ms (11)
  • GitHub Check: Check Rust code with rustfmt and clippy
  • GitHub Check: Check the WASM package and demo
  • GitHub Check: Run snippets and cpython tests on wasm-wasi
  • GitHub Check: Run tests under miri
  • GitHub Check: Run rust tests (windows-latest)
  • GitHub Check: Run snippets and cpython tests (macos-latest)
  • GitHub Check: Run rust tests (macos-latest)
  • GitHub Check: Run rust tests (ubuntu-latest)
  • GitHub Check: Run snippets and cpython tests (ubuntu-latest)
  • GitHub Check: Run snippets and cpython tests (windows-latest)
  • GitHub Check: Ensure compilation on various targets
🔇 Additional comments (2)
vm/src/stdlib/functools.rs (2)

265-323: Well-implemented repr with proper recursive handling and module qualification.

The repr implementation correctly handles all edge cases including recursive repr, proper formatting of keyword arguments, and appropriate module qualification for different contexts.


45-323: Excellent implementation of functools.partial with comprehensive feature coverage.

This implementation successfully provides:

  • Thread-safe access to partial state using PyRwLock
  • Correct handling of nested partials through flattening
  • Proper pickling support with reduce and setstate
  • Robust error handling with descriptive messages
  • Accurate repr formatting that matches Python's behavior

The removal of the @unittest.expectedFailure decorator in the test file confirms this implementation now passes all tests.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (3)
vm/src/stdlib/functools.rs (3)

103-138: State restoration has robust validation but consider simplifying tuple handling.

The validation logic is thorough, checking tuple length, callable validity, and type correctness. However, the tuple conversion logic for handling subclasses could be simplified.

Consider using a helper function for tuple conversion to reduce code duplication:

+            fn ensure_base_tuple(obj: &PyObjectRef, vm: &VirtualMachine) -> PyResult<PyRef<PyTuple>> {
+                match obj.clone().downcast::<PyTuple>() {
+                    Ok(tuple) if tuple.class().is(vm.ctx.types.tuple_type) => Ok(tuple),
+                    Ok(_) | Err(_) => {
+                        let elements: Vec<PyObjectRef> = obj.try_to_value(vm)?;
+                        Ok(vm.ctx.new_tuple(elements))
+                    }
+                }
+            }
+
-            // Always convert to base tuple, even if it's a subclass
-            let args_tuple = match args.clone().downcast::<PyTuple>() {
-                Ok(tuple) if tuple.class().is(vm.ctx.types.tuple_type) => tuple,
-                _ => {
-                    // It's a tuple subclass, convert to base tuple
-                    let elements: Vec<PyObjectRef> = args.try_to_value(vm)?;
-                    vm.ctx.new_tuple(elements)
-                }
-            };
+            let args_tuple = ensure_base_tuple(args, vm)?;

265-323: Representation implementation is comprehensive but could be optimized.

The representation logic handles recursion correctly and provides appropriate formatting. However, the qualified name logic could be simplified and the string handling could be more efficient.

Consider simplifying the qualified name logic and using more efficient string operations:

                // Check if this is a subclass by comparing with the base partial type
-                let is_subclass = !zelf.class().is(PyPartial::class(&vm.ctx));
-
-                let qualified_name = if !is_subclass {
+                let qualified_name = if zelf.class().is(PyPartial::class(&vm.ctx)) {
                    // For the base partial class, always use functools.partial
                    "functools.partial".to_string()
                } else {
                    // For subclasses, check if they're defined in __main__ or test modules
                    match module.downcast::<crate::builtins::PyStr>() {
                        Ok(module_str) => {
                            let module_name = module_str.as_str();
                            match module_name {
-                                "builtins" | "" | "__main__" => class_name.to_string(),
+                                "builtins" | "" | "__main__" => class_name.to_owned(),
                                name if name.starts_with("test.") || name == "test" => {
-                                    // For test modules, just use the class name without module prefix
-                                    class_name.to_string()
+                                    class_name.to_owned()
                                }
                                _ => format!("{}.{}", module_name, class_name),
                            }
                        }
-                        Err(_) => class_name.to_string(),
+                        Err(_) => class_name.to_owned(),
                    }
                };

275-289: Argument formatting logic could be more robust.

The representation of arguments and keywords is generally correct, but the string conversion for non-string keys could be handled more defensively.

Consider adding error handling for the string conversion:

                for (key, value) in inner.keywords.clone() {
                    // For string keys, use them directly without quotes
                    let key_part = if let Ok(s) = key.clone().downcast::<crate::builtins::PyStr>() {
                        s.as_str().to_owned()
                    } else {
                        // For non-string keys, convert to string using __str__
-                        key.str(vm)?.as_str().to_owned()
+                        match key.str(vm) {
+                            Ok(s) => s.as_str().to_owned(),
+                            Err(_) => "<unprintable key>".to_owned(),
+                        }
                    };
-                    let value_str = value.repr(vm)?;
+                    let value_str = value.repr(vm).unwrap_or_else(|_| vm.ctx.new_str("<unprintable>"));
                    parts.push(format!("{}={}", key_part, value_str.as_str()));
                }

This prevents representation failures from propagating and provides more user-friendly output.

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 960f87b and be05bc1.

📒 Files selected for processing (3)
  • Lib/test/test_functools.py (0 hunks)
  • Lib/test/test_threading.py (0 hunks)
  • vm/src/stdlib/functools.rs (2 hunks)
💤 Files with no reviewable changes (2)
  • Lib/test/test_threading.py
  • Lib/test/test_functools.py
🧰 Additional context used
📓 Path-based instructions (1)
`**/*.rs`: Follow the default rustfmt code style; use 'cargo fmt' to format Rust code. Always run clippy to lint Rust code ('cargo clippy') before completing tasks, and fix any war...

**/*.rs: Follow the default rustfmt code style; use 'cargo fmt' to format Rust code.
Always run clippy to lint Rust code ('cargo clippy') before completing tasks, and fix any warnings or lints introduced by your changes.

  • vm/src/stdlib/functools.rs
⏰ Context from checks skipped due to timeout of 90000ms (11)
  • GitHub Check: Run snippets and cpython tests on wasm-wasi
  • GitHub Check: Run tests under miri
  • GitHub Check: Check the WASM package and demo
  • GitHub Check: Run snippets and cpython tests (ubuntu-latest)
  • GitHub Check: Run snippets and cpython tests (macos-latest)
  • GitHub Check: Run rust tests (windows-latest)
  • GitHub Check: Check Rust code with rustfmt and clippy
  • GitHub Check: Run snippets and cpython tests (windows-latest)
  • GitHub Check: Run rust tests (ubuntu-latest)
  • GitHub Check: Run rust tests (macos-latest)
  • GitHub Check: Ensure compilation on various targets
🔇 Additional comments (6)
vm/src/stdlib/functools.rs (6)

5-16: Imports look comprehensive and well-organized.

The imports cover all necessary components for the implementation including thread safety (PyRwLock), function handling (FuncArgs, KwArgs), and various traits. Good use of the indexmap crate for maintaining key order in keyword arguments.


45-58: Struct design follows Rust best practices.

The use of PyRwLock<PyPartialInner> provides thread safety while allowing both read and write access. The inner struct cleanly separates the mutable state with appropriate types (PyObjectRef, PyRef<PyTuple>, PyRef<PyDict>).


61-74: Property getters are correctly implemented.

The getters properly acquire read locks and clone the references, which is safe and follows Rust conventions. The property names match Python's functools.partial interface.


76-101: Pickling implementation follows Python semantics correctly.

The __reduce__ method properly constructs the expected tuple format (type, (func,), state) where state contains (func, args, keywords, __dict__). The handling of the instance dictionary (__dict__) correctly checks for existence and emptiness before including it.


139-192: Dictionary handling and instance state restoration are well-implemented.

The logic correctly handles None values, dictionary subclasses, and instance dictionary updates. The approach of clearing and rebuilding the dictionary ensures clean state restoration.


195-233: Constructor correctly handles nested partial flattening.

The implementation properly validates the first argument is callable and correctly flattens nested partial objects by merging their arguments and keywords. The keyword handling uses intern_str which is appropriate for optimization.

Comment on lines 235 to 263
impl Callable for PyPartial {
type Args = FuncArgs;

fn call(zelf: &Py<Self>, args: FuncArgs, vm: &VirtualMachine) -> PyResult {
let inner = zelf.inner.read();
let mut combined_args = inner.args.as_slice().to_vec();
combined_args.extend_from_slice(&args.args);

// Merge keywords from self.keywords and args.kwargs
let mut final_kwargs = IndexMap::new();

// Add keywords from self.keywords
for (key, value) in inner.keywords.clone() {
let key_str = key
.downcast::<crate::builtins::PyStr>()
.map_err(|_| vm.new_type_error("keywords must be strings".to_owned()))?;
final_kwargs.insert(key_str.as_str().to_owned(), value);
}

// Add keywords from args.kwargs (these override self.keywords)
for (key, value) in args.kwargs {
final_kwargs.insert(key, value);
}

inner
.func
.call(FuncArgs::new(combined_args, KwArgs::new(final_kwargs)), vm)
}
}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Callable implementation has a potential issue with keyword validation.

The callable logic correctly merges arguments and keywords, but there's an inconsistency in keyword validation that could cause runtime errors.

The keyword validation in the callable implementation differs from the constructor. In the constructor, all keywords are converted to interned strings, but in the callable, existing keywords are validated to be strings and could fail:

            // Add keywords from self.keywords
            for (key, value) in inner.keywords.clone() {
-                let key_str = key
-                    .downcast::<crate::builtins::PyStr>()
-                    .map_err(|_| vm.new_type_error("keywords must be strings".to_owned()))?;
-                final_kwargs.insert(key_str.as_str().to_owned(), value);
+                // Keywords should already be strings from construction, but handle gracefully
+                let key_str = if let Ok(s) = key.downcast::<crate::builtins::PyStr>() {
+                    s.as_str().to_owned()
+                } else {
+                    // This shouldn't happen if construction is correct, but handle it
+                    key.str(vm)?.as_str().to_owned()
+                };
+                final_kwargs.insert(key_str, value);

This ensures consistency with the construction logic where keywords are always stored as strings.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
impl Callable for PyPartial {
type Args = FuncArgs;
fn call(zelf: &Py<Self>, args: FuncArgs, vm: &VirtualMachine) -> PyResult {
let inner = zelf.inner.read();
let mut combined_args = inner.args.as_slice().to_vec();
combined_args.extend_from_slice(&args.args);
// Merge keywords from self.keywords and args.kwargs
let mut final_kwargs = IndexMap::new();
// Add keywords from self.keywords
for (key, value) in inner.keywords.clone() {
let key_str = key
.downcast::<crate::builtins::PyStr>()
.map_err(|_| vm.new_type_error("keywords must be strings".to_owned()))?;
final_kwargs.insert(key_str.as_str().to_owned(), value);
}
// Add keywords from args.kwargs (these override self.keywords)
for (key, value) in args.kwargs {
final_kwargs.insert(key, value);
}
inner
.func
.call(FuncArgs::new(combined_args, KwArgs::new(final_kwargs)), vm)
}
}
impl Callable for PyPartial {
type Args = FuncArgs;
fn call(zelf: &Py<Self>, args: FuncArgs, vm: &VirtualMachine) -> PyResult {
let inner = zelf.inner.read();
let mut combined_args = inner.args.as_slice().to_vec();
combined_args.extend_from_slice(&args.args);
// Merge keywords from self.keywords and args.kwargs
let mut final_kwargs = IndexMap::new();
// Add keywords from self.keywords
for (key, value) in inner.keywords.clone() {
// Keywords should already be strings from construction, but handle gracefully
let key_str = if let Ok(s) = key.downcast::<crate::builtins::PyStr>() {
s.as_str().to_owned()
} else {
// This shouldn't happen if construction is correct, but handle it
key.str(vm)?.as_str().to_owned()
};
final_kwargs.insert(key_str, value);
}
// Add keywords from args.kwargs (these override self.keywords)
for (key, value) in args.kwargs {
final_kwargs.insert(key, value);
}
inner
.func
.call(FuncArgs::new(combined_args, KwArgs::new(final_kwargs)), vm)
}
}
🤖 Prompt for AI Agents
In vm/src/stdlib/functools.rs around lines 235 to 263, the call method in the
Callable implementation for PyPartial inconsistently validates keywords by
downcasting keys to PyStr and returning an error if they are not strings, unlike
the constructor which converts all keywords to interned strings. To fix this,
ensure that the callable method processes keywords the same way as the
constructor by converting all keyword keys to interned strings before merging,
maintaining consistency and preventing runtime errors.

@youknowone youknowone merged commit 0624ca6 into RustPython:main Jun 23, 2025
12 checks passed
@youknowone youknowone deleted the partial branch June 23, 2025 14:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant