micropython · andrewleech · Jun 5, 2025 · Jun 5, 2025 · Jun 6, 2025 · Jun 7, 2025
diff --git a/.gitmodules b/.gitmodules
@@ -74,3 +74,6 @@
 [submodule "lib/alif-security-toolkit"]
 	path = lib/alif-security-toolkit
 	url = https://github.com/micropython/alif-security-toolkit.git
+[submodule "lib/lvgl"]
+	path = lib/lvgl
+	url = https://github.com/lvgl/lv_binding_micropython.git
diff --git a/docs/develop/natmod-implementation-plan.md b/docs/develop/natmod-implementation-plan.md
@@ -0,0 +1,243 @@
+# Native Module Static Definition - Implementation Plan
+
+## Phase 1: Parser Tool Development
+
+### 1.1 Create `py/make_natmod_init.py`
+
+The parser tool will analyze C source files and generate initialization code.
+
+**Key Functions:**
+
+```python
+def find_module_registrations(source_files):
+    """Find MP_REGISTER_MODULE declarations in source files"""
+
+def extract_module_globals(source_content, module_obj_name):
+    """Extract the globals table for a given module object"""
+
+def generate_mpy_init(module_name, module_obj, globals_table):
+    """Generate the mpy_init function"""
+
+def process_native_module(source_files):
+    """Main processing function"""
+```
+
+**Generated Output Example:**
+
+```c
+// Auto-generated by make_natmod_init.py
+#include "py/dynruntime.h"
+
+// Forward declarations
+extern const mp_obj_module_t example_module;
+extern const mp_rom_map_elem_t example_globals_table[];
+
+mp_obj_t mpy_init(mp_obj_fun_bc_t *self, size_t n_args, size_t n_kw, mp_obj_t *args) {
+    MP_DYNRUNTIME_INIT_ENTRY
+
+    // Register module globals from static table
+    const mp_rom_map_elem_t *table = example_globals_table;
+    size_t table_size = /* extracted size */;
+
+    for (size_t i = 0; i < table_size; i++) {
+        qstr key = MP_OBJ_QSTR_VALUE(table[i].key);
+        if (key != MP_QSTR___name__) {
+            mp_store_global(key, table[i].value);
+        }
+    }
+
+    MP_DYNRUNTIME_INIT_EXIT
+}
+
+// Module name export for build system
+const char *natmod_module_name = "example";
+```
+
+### 1.2 Parser Implementation Details
+
+**Regular Expressions:**
+```python
+# Find MP_REGISTER_MODULE declarations
+register_pattern = re.compile(
+    r'MP_REGISTER_MODULE\s*\(\s*MP_QSTR_(\w+)\s*,\s*(\w+)\s*\)',
+    re.MULTILINE
+)
+
+# Find module object definitions
+module_obj_pattern = re.compile(
+    r'const\s+mp_obj_module_t\s+(\w+)\s*=\s*{[^}]+\.globals\s*=\s*\([^)]+\)&(\w+)',
+    re.DOTALL
+)
+
+# Find globals table definitions
+globals_table_pattern = re.compile(
+    r'static\s+const\s+mp_rom_map_elem_t\s+(\w+)\s*\[\s*\]\s*=\s*{([^}]+)}',
+    re.DOTALL
+)
+```
+
+## Phase 2: Build System Integration
+
+### 2.1 Update `py/dynruntime.mk`
+
+**Changes Required:**
+
+```makefile
+# Add parser tool path
+MAKE_NATMOD_INIT = $(PY_SRC)/make_natmod_init.py
+
+# Check if source uses static module definition
+STATIC_MODULE_SRC = $(shell grep -l "MP_REGISTER_MODULE" $(SRC) 2>/dev/null)
+
+ifneq ($(STATIC_MODULE_SRC),)
+# Use static module definition
+$(BUILD)/natmod_init_gen.c: $(SRC)
+	$(PYTHON) $(MAKE_NATMOD_INIT) $(SRC) > $@
+
+SRC_GEN = $(BUILD)/natmod_init_gen.c
+MOD = $(shell $(PYTHON) $(MAKE_NATMOD_INIT) --get-name $(SRC))
+else
+# Use traditional dynamic definition
+SRC_GEN =
+endif
+
+# Update object file list
+OBJ = $(SRC:.c=.o) $(SRC_GEN:.c=.o)
+```
+
+### 2.2 Backward Compatibility
+
+The build system will:
+1. Check if source files contain `MP_REGISTER_MODULE`
+2. If found, use the parser to generate init code
+3. If not found, use traditional build process
+4. Allow `MOD` override in Makefile for legacy modules
+
+## Phase 3: Runtime Support Extensions
+
+### 3.1 Extend `py/dynruntime.h`
+
+**New Macros:**
+
+```c
+// Helper to register static globals table
+#define MP_DYNRUNTIME_REGISTER_GLOBALS_TABLE(table, size) \
+    do { \
+        for (size_t i = 0; i < (size); i++) { \
+            qstr key = MP_OBJ_QSTR_VALUE((table)[i].key); \
+            if (key != MP_QSTR___name__) { \
+                mp_store_global(key, (table)[i].value); \
+            } \
+        } \
+    } while (0)
+
+// Optimized module attribute lookup for static modules
+#define MP_DYNRUNTIME_MODULE_STATIC_GETATTR(module_obj, attr, dest) \
+    mp_module_generic_getattr((module_obj), (attr), (dest))
+```
+
+### 3.2 Performance Optimizations
+
+Consider future optimizations:
+- Cache static module globals for faster lookup
+- Precompute attribute hash tables
+- Optimize common attribute access patterns
+
+## Phase 4: Testing Strategy
+
+### 4.1 Test Coverage
+
+1. **Parser Tests**
+   - Valid module definitions
+   - Multiple modules in one file
+   - Edge cases and malformed input
+
+2. **Build System Tests**
+   - Static module builds correctly
+   - Dynamic module still works
+   - Mixed projects supported
+
+3. **Runtime Tests**
+   - Module imports correctly
+   - All attributes accessible
+   - Performance benchmarks
+
+### 4.2 Example Test Module
+
+Create `examples/natmod/static_module/`:
+
+```c
+// static_module.c
+#include "py/dynruntime.h"
+
+static mp_obj_t test_function(mp_obj_t arg) {
+    return mp_obj_new_int(mp_obj_get_int(arg) * 2);
+}
+static MP_DEFINE_CONST_FUN_OBJ_1(test_function_obj, test_function);
+
+static const mp_rom_map_elem_t static_module_globals_table[] = {
+    { MP_ROM_QSTR(MP_QSTR___name__), MP_ROM_QSTR(MP_QSTR_static_module) },
+    { MP_ROM_QSTR(MP_QSTR_test), MP_ROM_PTR(&test_function_obj) },
+};
+static MP_DEFINE_CONST_DICT(static_module_globals, static_module_globals_table);
+
+const mp_obj_module_t static_module = {
+    .base = { &mp_type_module },
+    .globals = (mp_obj_dict_t *)&static_module_globals,
+};
+
+MP_REGISTER_MODULE(MP_QSTR_static_module, static_module);
+```
+
+## Phase 5: Migration Guide
+
+### 5.1 Converting Existing Modules
+
+**Before:**
+```c
+mp_obj_t mpy_init(...) {
+    MP_DYNRUNTIME_INIT_ENTRY
+    mp_store_global(MP_QSTR_func, MP_OBJ_FROM_PTR(&func_obj));
+    MP_DYNRUNTIME_INIT_EXIT
+}
+```
+
+**After:**
+```c
+static const mp_rom_map_elem_t module_globals_table[] = {
+    { MP_ROM_QSTR(MP_QSTR___name__), MP_ROM_QSTR(MP_QSTR_mymodule) },
+    { MP_ROM_QSTR(MP_QSTR_func), MP_ROM_PTR(&func_obj) },
+};
+static MP_DEFINE_CONST_DICT(module_globals, module_globals_table);
+
+const mp_obj_module_t mymodule = {
+    .base = { &mp_type_module },
+    .globals = (mp_obj_dict_t *)&module_globals,
+};
+
+MP_REGISTER_MODULE(MP_QSTR_mymodule, mymodule);
+```
+
+### 5.2 Best Practices
+
+1. Use static definition for new modules
+2. Migrate existing modules gradually
+3. Keep module structure consistent
+4. Document any module-specific initialization needs
+
+## Implementation Timeline
+
+1. **Week 1-2**: Parser tool development and testing
+2. **Week 3**: Build system integration
+3. **Week 4**: Runtime support and optimizations
+4. **Week 5**: Testing and documentation
+5. **Week 6**: Example migrations and refinement
+
+## Success Criteria
+
+- [ ] Parser correctly handles all existing module patterns
+- [ ] Build system seamlessly supports both approaches
+- [ ] No performance regression for existing modules
+- [ ] At least 3 example modules converted successfully
+- [ ] Documentation updated with new patterns
+- [ ] All existing tests pass without modification
diff --git a/docs/develop/natmod-static-modules.md b/docs/develop/natmod-static-modules.md
@@ -0,0 +1,123 @@
+# Native Module Static Definition System
+
+## Overview
+
+This document describes a proposed enhancement to MicroPython's native module (natmod) system that would allow native modules to use the same static module definition pattern as user C modules (usercmodule). This would reduce boilerplate code and improve consistency across the codebase.
+
+## Current State Analysis
+
+### User C Modules (usercmodule)
+
+User C modules use a static definition pattern:
+
+```c
+// Define module globals table
+static const mp_rom_map_elem_t module_globals_table[] = {
+    { MP_ROM_QSTR(MP_QSTR___name__), MP_ROM_QSTR(MP_QSTR_module_name) },
+    { MP_ROM_QSTR(MP_QSTR_function), MP_ROM_PTR(&function_obj) },
+    { MP_ROM_QSTR(MP_QSTR_Class), MP_ROM_PTR(&type_Class) },
+};
+static MP_DEFINE_CONST_DICT(module_globals, module_globals_table);
+
+// Define module object
+const mp_obj_module_t module_user_cmodule = {
+    .base = { &mp_type_module },
+    .globals = (mp_obj_dict_t *)&module_globals,
+};
+
+// Register module
+MP_REGISTER_MODULE(MP_QSTR_module_name, module_user_cmodule);
+```
+
+Key characteristics:
+- Module structure is statically defined at compile time
+- `MP_REGISTER_MODULE` macro is parsed by `py/makemoduledefs.py`
+- Module attributes are defined in a const table
+- No runtime initialization code needed
+
+### Native Modules (natmod)
+
+Native modules use a dynamic initialization pattern:
+
+```c
+// Module initialization function
+mp_obj_t mpy_init(mp_obj_fun_bc_t *self, size_t n_args, size_t n_kw, mp_obj_t *args) {
+    MP_DYNRUNTIME_INIT_ENTRY
+
+    // Register each attribute individually
+    mp_store_global(MP_QSTR_function, MP_OBJ_FROM_PTR(&function_obj));
+    mp_store_global(MP_QSTR_Class, MP_OBJ_FROM_PTR(&type_Class));
+
+    MP_DYNRUNTIME_INIT_EXIT
+}
+```
+
+Key characteristics:
+- Module name defined in Makefile (`MOD = module_name`)
+- Attributes registered dynamically in `mpy_init`
+- More boilerplate code required
+- Runtime initialization overhead
+
+## Problem Statement
+
+The current native module system has several limitations:
+
+1. **Inconsistency**: Different patterns for defining modules makes code harder to understand and maintain
+2. **Boilerplate**: Manual attribute registration is repetitive and error-prone
+3. **Maintenance**: Changes to module structure require updates in multiple places
+4. **Code Sharing**: Difficult to share module definitions between usercmodule and natmod
+
+## Proposed Solution
+
+### Goal
+
+Enable native modules to use the same static module definition pattern as user C modules while maintaining backward compatibility with existing native modules.
+
+### Approach
+
+1. **Code Generation**: Create a preprocessing tool that parses static module definitions and generates the required `mpy_init` function
+2. **Build System Integration**: Integrate the preprocessing step into the native module build process
+3. **Runtime Support**: Extend dynruntime to efficiently handle static module definitions
+4. **Backward Compatibility**: Support both old and new patterns transparently
+
+### Benefits
+
+- **Consistency**: Same module definition pattern across all module types
+- **Less Boilerplate**: Automatic generation of initialization code
+- **Better Maintenance**: Module structure defined in one place
+- **Code Reuse**: Easier to convert between module types
+- **Performance**: Potential optimizations for static definitions
+
+## Implementation Components
+
+### 1. Parser Tool (`py/make_natmod_init.py`)
+
+A Python script that:
+- Parses C source files for `MP_REGISTER_MODULE` declarations
+- Extracts module name and module object references
+- Locates associated globals tables
+- Generates appropriate `mpy_init` function
+
+### 2. Build System Updates (`py/dynruntime.mk`)
+
+Modifications to:
+- Add preprocessing step before compilation
+- Extract module name from source instead of Makefile
+- Handle generated initialization code
+
+### 3. Runtime Extensions (`py/dynruntime.h`)
+
+New macros and functions to:
+- Support static globals dictionary traversal
+- Optimize attribute access for static modules
+- Maintain compatibility with dynamic registration
+
+### 4. Migration Path
+
+- Existing native modules continue to work unchanged
+- New modules can use static definition pattern
+- Gradual migration of existing modules over time
+
+## Next Steps
+
+See [implementation-plan.md](natmod-implementation-plan.md) for detailed technical implementation steps.