-
-
Notifications
You must be signed in to change notification settings - Fork 8.3k
natmod: Add support for static module definitions in native modules #17461
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
andrewleech
wants to merge
5
commits into
micropython:master
Choose a base branch
from
andrewleech:natmod-static-module-definition
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
natmod: Add support for static module definitions in native modules #17461
andrewleech
wants to merge
5
commits into
micropython:master
from
andrewleech:natmod-static-module-definition
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Andrew Leech <[email protected]>
Signed-off-by: Andrew Leech <[email protected]>
This adds documentation for a proposed enhancement to allow native modules to use the same static module definition pattern as user C modules, reducing boilerplate and improving consistency. - natmod-static-modules.md: Overview and analysis of current state - natmod-implementation-plan.md: Detailed implementation strategy The proposal includes a parser tool to auto-generate init functions, build system integration, and maintains backward compatibility.
This commit introduces improvements to reduce boilerplate in native module definitions while maintaining compatibility: 1. Parser tool (py/make_natmod_init.py) - Generates init functions for modules using MP_REGISTER_MODULE pattern (experimental). 2. Build system integration - dynruntime.mk can auto-detect static module definitions and extract module names from source. 3. Helper macros in dynruntime.h: - MP_OBJ_QSTR_VALUE: Extract QSTR from object - MP_DYNRUNTIME_REGISTER_GLOBALS_TABLE: Register module globals - MP_DYNRUNTIME_INIT_STATIC_MODULE: Simplified registration 4. Working example (static_module_hybrid) demonstrates a practical macro-based approach that reduces boilerplate. The implementation recognizes native module constraints where QSTRs are runtime values, not compile-time constants. The hybrid approach provides a cleaner way to define module attributes while working within these constraints.
This commit adds infrastructure to support a cleaner way of defining native modules using static patterns similar to user C modules: 1. Added make_natmod_init.py parser tool to extract MP_REGISTER_MODULE declarations and generate mpy_init functions automatically. 2. Enhanced dynruntime.mk to auto-detect static module patterns and invoke the parser tool when needed. 3. Added helper macros to dynruntime.h: - MP_OBJ_QSTR_VALUE: Extract QSTR value from object at runtime - MP_DYNRUNTIME_REGISTER_GLOBALS_TABLE: Register globals from table - MP_DYNRUNTIME_INIT_STATIC_MODULE: Initialize module from definition 4. Created static_module_hybrid example demonstrating macro-based approach that reduces boilerplate while maintaining zero overhead. 5. Updated documentation in docs/develop/natmod.rst with new patterns and helper macros. 6. Added dual-mode module examples showing how the same C source can be compiled as either a user C module or native module. The traditional mpy_init approach remains fully supported. The new infrastructure is optional and provides a migration path for cleaner module definitions. Signed-off-by: Assistant <[email protected]>
Code size report:
|
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #17461 +/- ##
=======================================
Coverage 98.54% 98.54%
=======================================
Files 169 169
Lines 21943 21943
=======================================
Hits 21623 21623
Misses 320 320 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Note: This draft PR is pending manual review before it's ready for consideration.
This PR adds infrastructure to support a cleaner way of defining native modules using static patterns similar to user C modules. The traditional
mpy_init
approach remains fully supported - this provides an optional migration path for cleaner module definitions.Key Changes
Parser tool (
py/make_natmod_init.py
): ExtractsMP_REGISTER_MODULE
declarations and generatesmpy_init
functions automaticallyBuild system (
py/dynruntime.mk
): Auto-detects static module patterns and invokes parser when neededHelper macros (
py/dynruntime.h
):MP_OBJ_QSTR_VALUE
: Extract QSTR value from object at runtimeMP_DYNRUNTIME_REGISTER_GLOBALS_TABLE
: Register globals from tableMP_DYNRUNTIME_INIT_STATIC_MODULE
: Initialize module from definitionExamples:
static_module_hybrid
: Demonstrates macro-based approach with zero overheaddualmode
: Shows how same C source can compile as either user C module or native moduleDocumentation: Updated
docs/develop/natmod.rst
with new patternsTesting
Trade-offs and Alternatives
The macro-based approach for existing natmod examples was attempted but reverted due to QSTR runtime limitations in native modules. The static_module_hybrid example shows the working approach using compile-time macro expansion.