Added comprehensive support for authentication with workflows and custom commands:

• Workflows: Supports per-step identity configuration with identity field and --identity flag override
• Custom Commands: Supports identity configuration in command definition with --identity flag override
• Integration Tests: Added 8 integration tests demonstrating mock provider authentication end-to-end
• Mock Provider: Updated to set ATMOS_IDENTITY for testing
• Documentation: Added sections explaining authentication usage in workflows and custom commands

Changes

• Added identity field to WorkflowStep schema
• Added identity field to Command schema
• Implemented ExecuteWorkflow identity resolution with per-step and command-line overrides
• Added automatic --identity flag to all custom commands
• Updated mock provider to set ATMOS_IDENTITY environment variable
• Fixed stubAuthManager.PrepareShellEnvironment() to properly merge environment variables
• Created workflow integration tests with mock provider
• Created custom command integration tests with mock provider
• Updated documentation with authentication workflow examples

Test Results

• ✅ 8/8 integration tests passing
• ✅ All pkg/auth tests passing (66 tests)
• ✅ Code compiles successfully
• ✅ No regressions introduced

Testing

Run the following to verify the changes:

# Test workflows with authentication
go test -v ./internal/exec -run TestWorkflowIntegration

# Test custom commands with authentication  
go test -v ./cmd -run TestCustomCommandIntegration

# Test all auth functionality
go test -v ./pkg/auth

# Build to verify compilation
go build .

Added defer perf.Track() calls to public functions in the GCS Terraform backend implementation to comply with Atmos coding standards. Also updated the lintroller to exclude the internal/gcp package from perf.Track() linting rules to prevent an import cycle.

Changes

• Added perf.Track() to 7 public functions in internal/terraform_backend/terraform_backend_gcs.go
• Updated lintroller exclusions in tools/lintroller/rule_perf_track.go to exclude internal/gcp package
• Reason: import cycle would be created (internal/gcp → pkg/perf → pkg/schema → pkg/store → internal/gcp)

Test plan

• Lint passes without errors
• Code builds successfully
• All unit tests pass
• Pre-commit hooks pass

Summary by CodeRabbit

• Chores
  • Added performance tracking infrastructure to internal backend operations to monitor execution time.
  • Updated linter configuration to optimize dependency handling.

Note: These changes are internal infrastructure improvements with no visible impact to end-users.

Comprehensive audit and fix of unconditional test output that was creating walls of JSON/YAML and debug info in CI logs. All verbose output now respects test verbosity settings by using t.Cleanup() handlers with t.Failed() checks - output only appears when tests actually fail.

Root Issue: Tests were using fmt.Print*() and unconditional t.Log() to dump captured command output, terraform plans, schema validation results, and debug info. These bypass Go's test verbosity controls and always output to CI logs.

Solution: Wrapped all verbose output in t.Cleanup() handlers that only log when t.Failed() is true, following the pattern established in PR #1704.

What Changed

15 Test Files Fixed

Terraform Output Dumps:

• internal/exec/terraform_test.go - terraform plan output
• tests/cli_terraform_test.go - terraform apply stdout/stderr

Command Output Captures:

• cmd/root_test.go - command output
• tests/validate_schema_test.go - schema validation output

Config/Command Debugging:

• pkg/merge/merge_context_demo_test.go - error formatting demos
• pkg/config/command_merging_behavior_test.go - command structure debugging
• pkg/config/command_merge_core_test.go - command verification output

Provenance Parser Debug Output:

• pkg/provenance/yaml_parser_multiline_test.go - pathMap dumps
• pkg/provenance/yaml_parser_arrays_test.go - 5 instances of pathMap iteration logging

Pattern Applied

All unconditional output converted to:

t.Cleanup(func() {
    if t.Failed() {
        t.Logf("Debug info: %s", output)
    }
})

Impact

• ✅ CI logs dramatically quieter - no verbose output on successful test runs
• ✅ Debug info preserved - still shows when tests fail
• ✅ Consistent with PR #1704 - same pattern across entire codebase
• ✅ Fixes pre-existing issues - some code dated back to May 2025

Testing

• Code compiles without errors
• All changes follow established PR #1704 pattern
• 3 clean, focused commits

Related to PR #1704 (quiet test output on success).

🤖 Generated with Claude Code

Co-Authored-By: Claude [email protected]

Comprehensive MFA documentation, bug fixes, and credential precedence improvements for AWS IAM user authentication.

Problems Discovered

1. MFA ARN Ignored When !env Variables Empty (BUG)

User Report: MFA configured in YAML but no TOTP prompt appeared when !env variables were empty.

Root Cause:

• User had mfa_arn: "arn:aws:iam::...:mfa/device" in YAML
• User had access_key_id: !env AWS_ACCESS_KEY_ID (empty environment variable)
• When !env returned empty string, code fell back to keyring credentials
• Keyring credentials had no MFA ARN → MFA ARN from YAML completely ignored
• Result: No TOTP prompt despite MFA being configured

Fix: Implemented deep merge precedence that overrides keyring MFA ARN with YAML MFA ARN when using keyring credentials.

2. Excessive WARN Logs for Empty Environment Variables

User Report: 8+ WARN messages for every empty !env function result.

Root Cause: Empty environment variables are normal (not yet configured, different environment) but were logged as warnings.

Fix: Changed all empty value logs from WARN → DEBUG level.

3. Unclear Credential Precedence

Issue: Ambiguous rules for which credential source wins (YAML vs keyring).

Fix: Implemented clear per-field deep merge precedence with comprehensive tests.

Solutions Implemented

1. Deep Merge Credential Precedence

Clear Rules:

1. YAML Complete (both access_key_id and secret_access_key non-empty)

   • Use YAML entirely (access keys + MFA ARN from YAML)
   • Keyring ignored

2. YAML Empty (both keys empty or omitted)

   • Use keyring credentials as base
   • Override MFA ARN from YAML if present ← Fixes the bug
   • Allows version-controlled MFA config with keyring-stored secrets

3. YAML Partial (only one key present)

   • Error: Both keys must be provided or both empty

Recommended Pattern:

identities:
  prod-admin:
    kind: aws/user
    credentials:
      # Access keys in keyring (via atmos auth user configure)
      # MFA ARN in YAML (version controlled)
      mfa_arn: arn:aws:iam::123456789012:mfa/username
      region: us-east-1

Benefits:

• Secure local credential storage (keyring)
• Shared team MFA configuration (YAML)
• Clear separation of secrets vs config

2. Comprehensive Test Suite

Added 6 table-driven tests validating:

• ✅ All 3 precedence rules
• ✅ Error conditions (partial credentials)
• ✅ Empty !env variable handling (user's bug scenario)
• ✅ MFA ARN override behavior

3. Reduced Log Noise

Changed empty value logs in pkg/config/process_yaml.go:

• !env empty: WARN → DEBUG
• !exec empty: WARN → DEBUG
• !include empty: WARN → DEBUG
• !repo-root empty: WARN → DEBUG

4. Enhanced MFA Documentation

website/docs/cli/commands/auth/usage.mdx

• "Multi-Factor Authentication (MFA) for AWS" subsection
• Configuration examples (YAML, env var, keyring)
• Step-by-step guide to find MFA device ARN
• Authentication flow with TOTP prompt visualization
• Security model explanation
• Troubleshooting guidance

website/docs/cli/commands/auth/auth-user-configure.mdx

• Enhanced MFA section with detailed guides
• Three configuration methods explained
• Security considerations

pkg/auth/docs/PRD/PRD-Atmos-Auth.md

• Deep merge precedence rules documented
• Configuration options with examples
• 5-step authentication flow
• Implementation details with file references

Files Changed

Bug Fixes:

• pkg/auth/identities/aws/user.go: Implement deep merge precedence
• pkg/config/process_yaml.go: Change empty value warnings to debug

Tests:

• pkg/auth/identities/aws/user_test.go: Add 6 comprehensive precedence tests

Documentation:

• website/docs/cli/commands/auth/usage.mdx: MFA section added
• website/docs/cli/commands/auth/auth-user-configure.mdx: Enhanced MFA docs
• pkg/auth/docs/PRD/PRD-Atmos-Auth.md: Precedence rules + XDG path fix

Testing

✅ All 24 AWS user identity tests pass (6 new + 18 existing)
✅ Compiled successfully
✅ No breaking changes
✅ User's bug scenario validated in tests

Key Points

✅ Fixed MFA bug - YAML MFA ARN now works with keyring credentials
✅ **Clear p...

what

• Add Google Cloud Storage (GCS) backend support to !terraform.state Atmos YAML function
• Implement performance optimizations (client caching, retry logic, extended timeouts)
• Create unified Google Cloud authentication system for consistency across GCP services
• Update documentation with GCS backend usage examples and authentication methods

why

The !terraform.state YAML function allows reading the outputs (remote state) of components in Atmos stack manifests directly from the configured Terraform/OpenTofu backends.

Previously, the !terraform.state YAML function only supported:

• local (Terraform and OpenTofu)
• s3 (Terraform and OpenTofu)

This PR adds support for:

• gcs (Google Cloud Storage - Terraform and OpenTofu)

With GCS backend support, users can now leverage the high-performance !terraform.state function instead of the slower !terraform.output or !store functions when using Google Cloud Storage for Terraform state storage.

Implementation Details

GCS Backend Features

• Full Authentication Support: JSON credentials, service account file paths, and Google Application Default Credentials (ADC)
• Service Account Impersonation: Support for impersonate_service_account configuration
• Performance Optimizations:
  • Client caching to avoid recreating GCS clients for repeated operations
  • Retry logic with exponential backoff (up to 3 attempts) for transient failures
  • Extended timeouts (30 seconds) to match S3 backend performance
• Robust Error Handling: Graceful handling of missing state files and detailed error context
• Resource Management: Proper cleanup and explicit resource management

Usage

The GCS backend works seamlessly with existing !terraform.state syntax:

# Get the `output` of the `component` in the current stack
subnet_id: !terraform.state vpc private_subnet_id

# Get the `output` of the `component` in the provided `stack` 
vpc_id: !terraform.state vpc dev-us-east-1 vpc_id

# Get complex outputs using YQ expressions
first_subnet: !terraform.state vpc .private_subnet_ids[0]

GCS Backend Configuration

The GCS backend supports all standard Terraform GCS backend configurations:

# atmos.yaml
components:
  terraform:
    backend_type: gcs
    backend:
      gcs:
        bucket: "my-terraform-state-bucket"
        prefix: "terraform/state"
        
        # Authentication options (choose one):
        
        # Option 1: JSON credentials content
        credentials: |
          {
            "type": "service_account",
            "project_id": "my-project",
            ...
          }
          
        # Option 2: Service account file path  
        credentials: "/path/to/service-account.json"
        
        # Option 3: Use Application Default Credentials (ADC)
        # (no credentials field needed - uses environment/metadata)
        
        # Optional: Service account impersonation
        impersonate_service_account: "[email protected]"

Performance Benefits

Compared to !terraform.output, the !terraform.state function with GCS backend:

• ✅ No Terraform execution - Reads state directly from GCS
• ✅ No provider initialization - Skips all module and provider setup
• ✅ No varfile generation - Bypasses Terraform configuration preparation
• ✅ Cached clients - Reuses GCS clients for multiple operations
• ✅ Parallel execution - Multiple state reads can happen concurrently

Testing

• Comprehensive Test Suite: 100% test coverage for all new functionality
• Mock Implementations: Complete interface-based testing for GCS operations
• Authentication Testing: Validates all credential types and authentication flows
• Error Scenario Coverage: Tests for missing files, network failures, and invalid configurations
• Caching Validation: Ensures client caching works correctly across operations
• Retry Logic Testing: Validates exponential backoff and failure recovery

Backward Compatibility

• ✅ No breaking changes to existing configurations
• ✅ Existing backends (local, s3) remain unchanged
• ✅ Same function syntax - no new parameters or options required
• ✅ Graceful fallbacks - continues to work with !terraform.output and !store functions

Files Changed

Core Implementation

• internal/terraform_backend/terraform_backend_gcs.go - GCS backend implementation
• internal/terraform_backend/terraform_backend_gcs_test.go - Comprehensive test suite
• internal/terraform_backend/terraform_backend_registry.go - Register GCS backend
• internal/terraform_backend/terraform_backend_utils.go - Updated error messages

Unified Authentication System

• internal/gcp/auth.go - New unified Google Cloud authentication (created)
• internal/gcp/auth_test.go - Authentication tests (created)
• pkg/store/google_secret_manager_store.go - Updated to use unified auth
• internal/gcp_utils/gcp_utils.go - Removed (replaced by unified auth)

Configuration & Documentation

• internal/exec/terraform_generate_backend.go - GCS backend validation
• website/docs/core-concepts/stacks/yaml-functions/terraform.state.mdx - Updated documentation
• errors/errors.go - Added GCS-specific error types
• go.mod - Added GCS storage dependency

Migration Guide

For users currently using !terraform.output or !store with GCS-stored state:

Before (slower)

# Using !terraform.output (requires Terraform execution)
vpc_id: !terraform.output vpc dev-us-east-1 vpc_id

# Using !store (requires separate state management)  
vpc_id: !store google-secret-manager dev/vpc/vpc_id

After (faster)

# Using !terraform.state (direct GCS state access)
vpc_id: !terraform.state vpc dev-us-east-1 vpc_id

Simply update your backend configuration to use gcs and replace function calls - no other changes needed!

Summary by CodeRabbit

• New Features

  • GCS-backed Terraform state support and unified Google Cloud authentication integration.

• Bug Fixes

  • Stricter backend config validation with clearer error responses and updated supported-backends messaging.

• Tests

  • Comprehensive unit tests added for GCS backend behavior and GCP authentication handling.

This PR addresses multiple authentication issues when using Atmos in containerized environments with mounted credential files:

1. Auth Pre-Hook Error Propagation - Terraform execution now properly aborts when authentication fails (e.g., Ctrl+C during SSO)
2. AWS Credential Loading Strategy - New LoadAtmosManagedAWSConfig() function provides proper isolation while preserving Atmos-managed profile selection
3. Noop Keyring Validation - Container auth now properly isolated from external environment variables
4. Whoami with Noop Keyring - atmos auth whoami now works in containerized environments
5. Test Coverage - Added test to verify auth errors properly abort execution

Changes

1. Auth Pre-Hook Error Propagation (internal/exec/terraform.go:236)

• Problem: Errors from auth pre-hook were logged but not returned, causing terraform execution to continue even when authentication failed (e.g., user presses Ctrl+C during SSO)
• Fix: Added return err after logging auth pre-hook errors
• Impact: Terraform commands now properly abort on auth failures

2. AWS Credential Loading Strategy (pkg/auth/cloud/aws/env.go)

• Problem: SDK's default config loading allowed IMDS access and was affected by external AWS_PROFILE, causing conflicts in containers
• Solution: Created LoadAtmosManagedAWSConfig() function that:
  • Clears credential env vars (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN)
  • Preserves profile/path vars (AWS_PROFILE, AWS_SHARED_CREDENTIALS_FILE, AWS_CONFIG_FILE)
  • Allows SDK to load from Atmos-managed credential files
• Impact: Proper isolation while still using Atmos-managed profiles

3. Noop Keyring Credential Validation (pkg/auth/credentials/keyring_noop.go)

• Problem: Used unrestricted config.LoadDefaultConfig() which allowed IMDS access and was affected by external AWS_PROFILE
• Fix: Changed to use LoadAtmosManagedAWSConfig()
• Impact: Container auth now properly isolated from external env vars

4. Whoami with Noop Keyring (pkg/auth/manager.go)

• Problem: Whoami() expected credentials from keyring, but noop keyring returns ErrCredentialsNotFound by design
• Fix: Added check for ErrCredentialsNotFound and fallback to buildWhoamiInfoFromEnvironment()
• Impact: atmos auth whoami now works in containerized environments

5. Test Coverage (internal/exec/terraform_test.go)

• Added TestExecuteTerraform_AuthPreHookErrorPropagation to verify auth errors properly abort execution
• Test validates that terraform doesn't continue on auth failure
• Updated test fixture to include required name_pattern configuration

Technical Details

The key insight is that Atmos sets AWS_PROFILE=identity-name (in pkg/auth/cloud/aws/setup.go:59) but the previous isolation approach cleared ALL AWS env vars including AWS_PROFILE. This caused the SDK to look for a non-existent [default] section.

The new LoadAtmosManagedAWSConfig preserves `AWS_...

what

• Add Google Cloud Storage (GCS) backend support to !terraform.state Atmos YAML function
• Implement performance optimizations (client caching, retry logic, extended timeouts)
• Create unified Google Cloud authentication system for consistency across GCP services
• Update documentation with GCS backend usage examples and authentication methods

why

The !terraform.state YAML function allows reading the outputs (remote state) of components in Atmos stack manifests directly from the configured Terraform/OpenTofu backends.

Previously, the !terraform.state YAML function only supported:

• local (Terraform and OpenTofu)
• s3 (Terraform and OpenTofu)

This PR adds support for:

• gcs (Google Cloud Storage - Terraform and OpenTofu)

With GCS backend support, users can now leverage the high-performance !terraform.state function instead of the slower !terraform.output or !store functions when using Google Cloud Storage for Terraform state storage.

Implementation Details

GCS Backend Features

• Full Authentication Support: JSON credentials, service account file paths, and Google Application Default Credentials (ADC)
• Service Account Impersonation: Support for impersonate_service_account configuration
• Performance Optimizations:
  • Client caching to avoid recreating GCS clients for repeated operations
  • Retry logic with exponential backoff (up to 3 attempts) for transient failures
  • Extended timeouts (30 seconds) to match S3 backend performance
• Robust Error Handling: Graceful handling of missing state files and detailed error context
• Resource Management: Proper cleanup and explicit resource management

Usage

The GCS backend works seamlessly with existing !terraform.state syntax:

# Get the `output` of the `component` in the current stack
subnet_id: !terraform.state vpc private_subnet_id

# Get the `output` of the `component` in the provided `stack` 
vpc_id: !terraform.state vpc dev-us-east-1 vpc_id

# Get complex outputs using YQ expressions
first_subnet: !terraform.state vpc .private_subnet_ids[0]

GCS Backend Configuration

The GCS backend supports all standard Terraform GCS backend configurations:

# atmos.yaml
components:
  terraform:
    backend_type: gcs
    backend:
      gcs:
        bucket: "my-terraform-state-bucket"
        prefix: "terraform/state"
        
        # Authentication options (choose one):
        
        # Option 1: JSON credentials content
        credentials: |
          {
            "type": "service_account",
            "project_id": "my-project",
            ...
          }
          
        # Option 2: Service account file path  
        credentials: "/path/to/service-account.json"
        
        # Option 3: Use Application Default Credentials (ADC)
        # (no credentials field needed - uses environment/metadata)
        
        # Optional: Service account impersonation
        impersonate_service_account: "[email protected]"

Performance Benefits

Compared to !terraform.output, the !terraform.state function with GCS backend:

• ✅ No Terraform execution - Reads state directly from GCS
• ✅ No provider initialization - Skips all module and provider setup
• ✅ No varfile generation - Bypasses Terraform configuration preparation
• ✅ Cached clients - Reuses GCS clients for multiple operations
• ✅ Parallel execution - Multiple state reads can happen concurrently

Testing

• Comprehensive Test Suite: 100% test coverage for all new functionality
• Mock Implementations: Complete interface-based testing for GCS operations
• Authentication Testing: Validates all credential types and authentication flows
• Error Scenario Coverage: Tests for missing files, network failures, and invalid configurations
• Caching Validation: Ensures client caching works correctly across operations
• Retry Logic Testing: Validates exponential backoff and failure recovery

Backward Compatibility

• ✅ No breaking changes to existing configurations
• ✅ Existing backends (local, s3) remain unchanged
• ✅ Same function syntax - no new parameters or options required
• ✅ Graceful fallbacks - continues to work with !terraform.output and !store functions

Files Changed

Core Implementation

• internal/terraform_backend/terraform_backend_gcs.go - GCS backend implementation
• internal/terraform_backend/terraform_backend_gcs_test.go - Comprehensive test suite
• internal/terraform_backend/terraform_backend_registry.go - Register GCS backend
• internal/terraform_backend/terraform_backend_utils.go - Updated error messages

Unified Authentication System

• internal/gcp/auth.go - New unified Google Cloud authentication (created)
• internal/gcp/auth_test.go - Authentication tests (created)
• pkg/store/google_secret_manager_store.go - Updated to use unified auth
• internal/gcp_utils/gcp_utils.go - Removed (replaced by unified auth)

Configuration & Documentation

• internal/exec/terraform_generate_backend.go - GCS backend validation
• website/docs/core-concepts/stacks/yaml-functions/terraform.state.mdx - Updated documentation
• errors/errors.go - Added GCS-specific error types
• go.mod - Added GCS storage dependency

Migration Guide

For users currently using !terraform.output or !store with GCS-stored state:

Before (slower)

# Using !terraform.output (requires Terraform execution)
vpc_id: !terraform.output vpc dev-us-east-1 vpc_id

# Using !store (requires separate state management)  
vpc_id: !store google-secret-manager dev/vpc/vpc_id

After (faster)

# Using !terraform.state (direct GCS state access)
vpc_id: !terraform.state vpc dev-us-east-1 vpc_id

Simply update your backend configuration to use gcs and replace function calls - no other changes needed!

Summary by CodeRabbit

• New Features

  • GCS-backed Terraform state support and unified Google Cloud authentication integration.

• Bug Fixes

  • Stricter backend config validation with clearer error responses and updated supported-backends messaging.

• Tests

  • Comprehensive unit tests added for GCS backend behavior and GCP authentication handling.

This PR addresses multiple authentication issues when using Atmos in containerized environments with mounted credential files:

1. Auth Pre-Hook Error Propagation - Terraform execution now properly aborts when authentication fails (e.g., Ctrl+C during SSO)
2. AWS Credential Loading Strategy - New LoadAtmosManagedAWSConfig() function provides proper isolation while preserving Atmos-managed profile selection
3. Noop Keyring Validation - Container auth now properly isolated from external environment variables
4. Whoami with Noop Keyring - atmos auth whoami now works in containerized environments
5. Test Coverage - Added test to verify auth errors properly abort execution

Changes

1. Auth Pre-Hook Error Propagation (internal/exec/terraform.go:236)

• Problem: Errors from auth pre-hook were logged but not returned, causing terraform execution to continue even when authentication failed (e.g., user presses Ctrl+C during SSO)
• Fix: Added return err after logging auth pre-hook errors
• Impact: Terraform commands now properly abort on auth failures

2. AWS Credential Loading Strategy (pkg/auth/cloud/aws/env.go)

• Problem: SDK's default config loading allowed IMDS access and was affected by external AWS_PROFILE, causing conflicts in containers
• Solution: Created LoadAtmosManagedAWSConfig() function that:
  • Clears credential env vars (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN)
  • Preserves profile/path vars (AWS_PROFILE, AWS_SHARED_CREDENTIALS_FILE, AWS_CONFIG_FILE)
  • Allows SDK to load from Atmos-managed credential files
• Impact: Proper isolation while still using Atmos-managed profiles

3. Noop Keyring Credential Validation (pkg/auth/credentials/keyring_noop.go)

• Problem: Used unrestricted config.LoadDefaultConfig() which allowed IMDS access and was affected by external AWS_PROFILE
• Fix: Changed to use LoadAtmosManagedAWSConfig()
• Impact: Container auth now properly isolated from external env vars

4. Whoami with Noop Keyring (pkg/auth/manager.go)

• Problem: Whoami() expected credentials from keyring, but noop keyring returns ErrCredentialsNotFound by design
• Fix: Added check for ErrCredentialsNotFound and fallback to buildWhoamiInfoFromEnvironment()
• Impact: atmos auth whoami now works in containerized environments

5. Test Coverage (internal/exec/terraform_test.go)

• Added TestExecuteTerraform_AuthPreHookErrorPropagation to verify auth errors properly abort execution
• Test validates that terraform doesn't continue on auth failure
• Updated test fixture to include required name_pattern configuration

Technical Details

The key insight is that Atmos sets AWS_PROFILE=identity-name (in pkg/auth/cloud/aws/setup.go:59) but the previous isolation approach cleared ALL AWS env vars including AWS_PROFILE. This caused the SDK to look for a non-existent [default] section.

The new LoadAtmosManagedAWSConfig preserves AWS_PROFILE while still preventing external credential conflicts.

...

Fixes stack configuration requirement for 6 commands that don't actually operate on stack manifests. These commands were incorrectly requiring stacks.base_path and stacks.included_paths to be configured, causing errors like:

Error: failed to initialize atmos config
stack base path must be provided in 'stacks.base_path' config or ATMOS_STACKS_BASE_PATH' ENV variable

What

Updated 6 commands to use processStacks=false in InitCliConfig:

Auth Commands (Commit 1)

• atmos auth env - Export cloud credentials as environment variables
• atmos auth exec - Execute commands with cloud credentials
• atmos auth shell - Launch authenticated shell

List/Docs Commands (Commit 2)

• atmos list workflows - List workflows from workflows/ directory
• atmos list vendor - List vendor configurations from component.yaml files
• atmos docs <component> - Display component README files

Why

These commands only need:

• Auth configuration from atmos.yaml
• Component base paths (terraform, helmfile, etc.)
• Workflow or vendor configurations

They do NOT need:

• Stack manifests to exist
• stacks.base_path to be configured
• stacks.included_paths to be configured

This makes Atmos more flexible for use cases like:

• CI/CD pipelines that only need auth or vendor management
• Development environments without full stack setup
• Documentation browsing without infrastructure configs
• Workflow management separate from stack operations

Technical Details

Changes Made

1. InitCliConfig parameter: Changed processStacks from true to false

   • Prevents validation requiring stacks.base_path and stacks.included_paths
   • Skips processing of stack manifest files

2. checkAtmosConfig option (for list vendor only): Added WithStackValidation(false)

   • Prevents checking if stacks directory exists
   • Required because list vendor calls checkAtmosConfig() with additional validation

Files Changed

• cmd/auth_env.go
• cmd/auth_exec.go
• cmd/auth_shell.go
• cmd/list_workflows.go
• cmd/list_vendor.go
• cmd/docs.go

Commands That Still Require Stacks (Unchanged)

These were NOT modified because they genuinely need stack manifests:

• atmos list stacks
• atmos list components
• atmos list settings
• atmos list values
• atmos list metadata

Testing

✅ All existing tests pass
✅ Linter passes with 0 issues
✅ Pre-commit hooks pass
✅ Manual testing confirms commands work without stack directories
✅ No regressions in existing functionality

References

Addresses user issue where atmos auth exec -- aws sts get-caller-identity failed with stack configuration error.

🤖 Generated with Claude Code

Co-Authored-By: Claude [email protected]

Summary by CodeRabbit

• New Features

  • Auth and utility commands (auth env, auth exec, auth shell, list workflows, list vendor, docs) now run without requiring stack configuration, enabling use in CI/CD, vendor management, and documentation workflows.

• Documentation

  • Added a blog post describing the change, usage examples, migration tips, and CI/CD benefits.

why

• Go releaser need more disk space

Summary by CodeRabbit

• Chores
  • Updated GitHub Actions runner specifications across feature release, nightly build, and test workflows to standardize build infrastructure configuration.

why

• Default runners have out of space

Summary by CodeRabbit

• Chores
  • Updated nightly release workflow to change how runner selection is provided: the workflow now accepts a JSON-like array of runner specifications, improving and broadening which runner(s) can be targeted for nightly builds.

why

• Terraform state operations need proper AWS credentials when accessing S3 backends
• Multi-identity scenarios require passing auth context through the call chain
• Reduces function parameter count from 6 to 2 (using PostAuthenticateParams struct)
• Simplifies nested conditional logic for better maintainability

references

• Part of multi-identity authentication context work
• Follows established authentication context patterns
• Related to docs/prd/auth-context-multi-identity.md

Summary by CodeRabbit

• New Features

  • Centralized per-command AuthContext enabling multiple concurrent identities (AWS, GitHub, Azure, etc.) and making in-process SDK and Terraform calls use Atmos-managed credentials.
  • Console session duration configurable via provider console.session_duration with CLI flag override.

• Bug Fixes

  • More reliable in-process authentication for SDK and Terraform state reads.

• Documentation

  • Added design doc, blog post, and CLI docs describing AuthContext and session-duration behavior.

• Tests

  • Expanded tests for auth flows, AWS config loading, and YAML/Terraform tag auth propagation.

why

• After commit 9fd7d15 (PR #1543), workflow shell commands lost access to PATH environment variable
• Users reported workflows that worked in v1.189.0 failed in v1.195.0 with commands like env, ls, grep not found
• This is a critical regression affecting any workflow using external executables
• Original fix conditionally replaced environment, which was inconsistent with executeCustomCommand behavior

Root Cause

The bug occurred in ExecuteShell() function in internal/exec/shell_utils.go:

1. Workflow commands call ExecuteShell with empty env slice: []string{}
2. ExecuteShell appends ATMOS_SHLVL to the slice: []string{"ATMOS_SHLVL=1"}
3. ShellRunner receives a non-empty env, so it doesn't fall back to os.Environ()
4. Shell command runs with ONLY ATMOS_SHLVL set, losing PATH and all other environment variables

Solution

Refactored ExecuteShell() to always merge custom env vars with parent environment:

// Always start with parent environment
mergedEnv := os.Environ()

// Merge custom env vars (overriding duplicates)
for _, envVar := range env {
    mergedEnv = u.UpdateEnvVar(mergedEnv, key, value)
}

// Add ATMOS_SHLVL
mergedEnv = append(mergedEnv, fmt.Sprintf("ATMOS_SHLVL=%d", newShellLevel))

This ensures:

• ✅ Empty env (workflows): Full parent environment including PATH
• ✅ Custom env (commands): Custom vars override parent, but PATH is preserved
• ✅ Consistent behavior: Matches executeCustomCommand pattern (line 393 in cmd_utils.go)

Testing

Unit Tests (internal/exec/shell_utils_test.go):

• TestExecuteShell/empty_env_should_inherit_PATH_from_parent_process - Verifies env command works
• TestExecuteShell/empty_env_should_inherit_PATH_for_common_commands - Tests ls, env, pwd, echo
• TestExecuteShell/custom_env_vars_override_parent_env - Verifies custom vars properly override parent

Integration Test (tests/test-cases/workflows.yaml):

• atmos workflow shell command with PATH - Full end-to-end workflow test using env | grep PATH

All tests pass, including existing workflow tests.

references

• Closes #1718
• Resolves DEV-3725
• Regression introduced in: 9fd7d15 (PR #1543)

Summary by CodeRabbit

• Bug Fixes

  • Shell commands now correctly inherit environment variables (including PATH) from the parent process, with custom env vars properly overriding parent values.

• Tests

  • Added tests covering environment inheritance for commands that require PATH, shell builtins, and custom env var overrides.

• Workflows / Snapshots

  • Added a workflow demonstrating PATH-dependent shell commands and updated related test snapshots and test cases.