Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@cidrblock
Copy link
Contributor

@cidrblock cidrblock commented Aug 17, 2025

Add environment variable support for CLI flags and refactor utilities module

Summary

This PR adds environment variable support for the --report and --command-borders CLI flags with precedence order defaults → env_file → os.environ → CLI args. It also moves util.py and boolean.py into a src/molecule/utils/ package.

User-Facing Changes

Environment Variable Support

New environment variables:

  • MOLECULE_REPORT=true - equivalent to --report
  • MOLECULE_COMMAND_BORDERS=false - equivalent to --command-borders

Environment variables are applied only when CLI arguments are not explicitly provided:

export MOLECULE_REPORT=true
molecule test                    # Uses env var

MOLECULE_REPORT=true molecule test --no-report  # CLI overrides env var

echo "MOLECULE_REPORT: true" > .env.yml
molecule test --env-file .env.yml               # Reads from env file

Supported boolean formats: true/false, yes/no, on/off, 1/0 (case-insensitive).

Implementation Details

Design Decision: New Pattern vs Existing

Molecule already supports some environment variables (MOLECULE_DEBUG, MOLECULE_VERBOSITY) but uses a pattern that has limitations:

# Existing pattern
MOLECULE_DEBUG: bool = boolean(os.environ.get("MOLECULE_DEBUG", "False"))
@click.option("--debug/--no-debug", default=MOLECULE_DEBUG, ...)

Issues with existing pattern:

  • No CLI override detection (env vars become Click defaults)
  • Module-level evaluation (no runtime flexibility)
  • No env file support (only reads os.environ)
  • Requires individual changes per environment variable

New pattern:

def _apply_env_overrides(self) -> None:
    merged_env = util.merge_dicts(os.environ, self.env)
    merged_env = set_env_from_file(merged_env, self.env_file)
    
    for env_var, config in ENV_VAR_CONFIG_MAPPING.items():
        if ctx.get_parameter_source(config_attr) == click.core.ParameterSource.COMMANDLINE:
            continue  # CLI takes precedence
        # Apply env var...

Improvements:

  • Proper CLI precedence via Click's parameter source detection
  • Full env file integration using existing set_env_from_file()
  • Runtime evaluation during config initialization
  • Extensible mapping-based configuration
  • Type-safe conversion

Environment Variable Processing

  1. Builds merged environment from os.environ, config.env, and env files
  2. Uses ctx.get_parameter_source() to detect explicit CLI arguments
  3. Applies type conversion based on ENV_VAR_CONFIG_MAPPING
  4. Runs before _apply_cli_overrides() to maintain precedence

Configuration mapping:

class EnvVarConfig(TypedDict):
    attr: Literal["report", "command_borders"]
    type: type

ENV_VAR_CONFIG_MAPPING: dict[str, EnvVarConfig] = {
    "MOLECULE_REPORT": {"attr": "report", "type": bool},
    "MOLECULE_COMMAND_BORDERS": {"attr": "command_borders", "type": bool},
}

Circular Import Resolution

Initial circular dependency: ansi_output.py → util.py → console.py → ansi_output.py

Resolution:

  1. Moved to_bool() to dedicated boolean.py module
  2. Organized both files into src/molecule/utils/ package
  3. Updated all imports to use explicit paths: from molecule.utils.boolean import to_bool

Module Refactoring

  • src/molecule/util.pysrc/molecule/utils/util.py
  • src/molecule/boolean.pysrc/molecule/utils/boolean.py
  • Added minimal src/molecule/utils/__init__.py

Testing

Added comprehensive test coverage for environment variable application, CLI precedence validation, and type conversion. All existing tests continue to pass.

Future Migration Path

Other environment variables (MOLECULE_DEBUG, MOLECULE_VERBOSITY, MOLECULE_PARALLEL) could be migrated to this new framework to gain:

  • Consistent CLI precedence handling
  • Env file support
  • Extensible configuration
  • Type-safe conversion

This would require updating their Click option definitions and adding entries to ENV_VAR_CONFIG_MAPPING.

… module

- Add environment variable support for --report and --command-borders CLI flags
- Implement precedence order: defaults → env vars → CLI args
- Add MOLECULE_REPORT and MOLECULE_COMMAND_BORDERS environment variables
- Move util.py and boolean.py into src/molecule/utils/ package for better organization
- Update all imports across codebase to use new utils package structure
- Add comprehensive test coverage for environment variable functionality
- Fix pre-commit linting issues and update import paths
- Environment variables now read from both os.environ and env_file
- Uses same merged environment pattern as existing Molecule config system
- Maintains proper precedence: defaults → env_file → os.environ → CLI args
- Replace unittest.mock.patch with monkeypatch.setenv/delenv
- Add proper pytest.MonkeyPatch type annotation
- Maintain standalone test capability with mock implementation
- Delete test_env_file.py and test_env_file_support.py
- Remove test_env_overrides_with_env_file from test_base.py
- Env file support is validated through existing comprehensive tests
- Focus on clean, maintainable test patterns consistent with codebase
- Remove test_env_overrides_with_env_file function that didn't follow patterns
- This was the source of the pydoclint DOC101/DOC103 errors
@cidrblock cidrblock force-pushed the feature_report_border_env branch from f9e4785 to 9058b76 Compare August 17, 2025 16:12
@cidrblock cidrblock changed the title Add environment variable support for CLI flags and refactor utilities module Add environment variable support for report and command-borders Aug 17, 2025
- Create src/molecule/util.py stub for backward compatibility
- Third-party plugins importing 'from molecule import util' will continue to work
- Issues deprecation warning to encourage migration to new location
- Fixes import errors in molecule-plugins-azure and other external drivers
- Add pylint disable for wildcard import warnings
- This is acceptable for a compatibility shim that needs to expose all functions
- Test validates that invalid environment variable values are logged and ignored
- Covers the error handling code paths in _apply_env_overrides method
- Uses int conversion to trigger ValueError for comprehensive coverage
- Move ENV_VAR_CONFIG_MAPPING import to module level for better code organization
- Follows Python import conventions by keeping imports at the top
- Use monkeypatch.setattr instead of manual dict manipulation
- Patch both constants and config module imports
- Remove try/finally cleanup as monkeypatch handles restoration automatically
- Add type: ignore comments to suppress TypedDict warnings
@cidrblock cidrblock merged commit 18a33a4 into ansible:main Aug 18, 2025
21 checks passed
Qalthos pushed a commit that referenced this pull request Aug 20, 2025
This PR consolidates the utils module structure by moving all utilities
into a single util.py file at the root of the molecule package.

Changes:
- Move util.py from src/molecule/utils/ to src/molecule/
- Consolidate to_bool function from boolean.py into util.py  
- Update all imports from molecule.utils.util to molecule.util
- Remove empty utils directory and associated files
- All existing functionality preserved and tested

This simplifies the module structure while maintaining backward
compatibility for all utility functions.

Reverts directory creation in #4530 now that circular imports issues
have been resolved.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

2 participants