Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@osterman
Copy link
Member

@osterman osterman commented Oct 22, 2025

what

  • Replace unconditional t.Log() calls with t.Cleanup() handlers that only output verbose YAML/data when tests fail
  • Eliminate noisy stderr output during successful test runs while preserving debug information when tests fail
  • Add fallback to raw data output (%+v) when YAML conversion produces empty strings

why

  • CI test runs were showing verbose YAML dumps to stderr even when tests passed
  • This cluttered test output and made it difficult to identify actual issues
  • Debug information is still valuable when tests fail, but shouldn't appear during successful runs
  • Go's t.Log() always outputs to stderr, regardless of test success/failure

demo

Finally clean output!

go mod download
Running tests with subprocess coverage collection
ok  	github.com/cloudposse/atmos	7.020s	coverage: 14.8% of statements in ./...
ok  	github.com/cloudposse/atmos/cmd	7.581s	coverage: 20.7% of statements in ./...
ok  	github.com/cloudposse/atmos/cmd/about	0.134s	coverage: 0.1% of statements in ./...
ok  	github.com/cloudposse/atmos/cmd/internal	0.099s	coverage: 0.1% of statements in ./...
?   	github.com/cloudposse/atmos/cmd/markdown	[no test files]
ok  	github.com/cloudposse/atmos/cmd/version	1.802s	coverage: 1.4% of statements in ./...
ok  	github.com/cloudposse/atmos/errors	0.213s	coverage: 0.4% of statements in ./...
ok  	github.com/cloudposse/atmos/internal/aws_utils	0.120s	coverage: 0.1% of statements in ./...
ok  	github.com/cloudposse/atmos/internal/exec	84.175s	coverage: 32.9% of statements in ./...
ok  	github.com/cloudposse/atmos/internal/terraform_backend	32.223s	coverage: 0.9% of statements in ./...
	github.com/cloudposse/atmos/internal/tui/atmos		coverage: 0.0% of statements
	github.com/cloudposse/atmos/internal/tui/components/code_view		coverage: 0.0% of statements
ok  	github.com/cloudposse/atmos/internal/tui/templates	0.125s	coverage: 0.5% of statements in ./...
	github.com/cloudposse/atmos/internal/tui/templates/term		coverage: 0.0% of statements
ok  	github.com/cloudposse/atmos/internal/tui/utils	0.218s	coverage: 0.2% of statements in ./...
	github.com/cloudposse/atmos/internal/tui/workflow		coverage: 0.0% of statements
ok  	github.com/cloudposse/atmos/pkg/atlantis	1.434s	coverage: 10.3% of statements in ./...
ok  	github.com/cloudposse/atmos/pkg/auth	0.141s	coverage: 2.1% of statements in ./...
ok  	github.com/cloudposse/atmos/pkg/auth/cloud/aws	0.113s	coverage: 0.8% of statements in ./...
ok  	github.com/cloudposse/atmos/pkg/auth/credentials	0.316s	coverage: 0.9% of statements in ./...
ok  	github.com/cloudposse/atmos/pkg/auth/factory	0.141s	coverage: 0.2% of statements in ./...
ok  	github.com/cloudposse/atmos/pkg/auth/identities/aws	0.139s	coverage: 1.7% of statements in ./...
ok  	github.com/cloudposse/atmos/pkg/auth/list	0.138s	coverage: 1.5% of statements in ./...
ok  	github.com/cloudposse/atmos/pkg/auth/providers/aws	0.098s	coverage: 1.6% of statements in ./...
ok  	github.com/cloudposse/atmos/pkg/auth/providers/github	0.072s	coverage: 0.3% of statements in ./...
ok  	github.com/cloudposse/atmos/pkg/auth/providers/mock	0.133s	coverage: 0.1% of statements in ./...
ok  	github.com/cloudposse/atmos/pkg/auth/types	0.075s	coverage: 0.2% of statements in ./...
ok  	github.com/cloudposse/atmos/pkg/auth/utils	0.099s	coverage: 0.0% of statements in ./...
ok  	github.com/cloudposse/atmos/pkg/auth/validation	0.150s	coverage: 0.7% of statements in ./...
ok  	github.com/cloudposse/atmos/pkg/aws	0.199s	coverage: 2.4% of statements in ./...
ok  	github.com/cloudposse/atmos/pkg/component	0.898s	coverage: 10.1% of statements in ./...
ok  	github.com/cloudposse/atmos/pkg/component/mock	0.178s	coverage: 0.4% of statements in ./...
ok  	github.com/cloudposse/atmos/pkg/config	3.247s	coverage: 5.7% of statements in ./...
ok  	github.com/cloudposse/atmos/pkg/config/homedir	0.073s	coverage: 0.2% of statements in ./...
ok  	github.com/cloudposse/atmos/pkg/convert	0.048s	coverage: 0.0% of statements in ./...
ok  	github.com/cloudposse/atmos/pkg/datafetcher	0.228s	coverage: 0.2% of statements in ./...
ok  	github.com/cloudposse/atmos/pkg/describe	29.214s	coverage: 13.2% of statements in ./...
ok  	github.com/cloudposse/atmos/pkg/downloader	1.115s	coverage: 1.6% of statements in ./...
ok  	github.com/cloudposse/atmos/pkg/filematch	0.135s	coverage: 0.3% of statements in ./...
	github.com/cloudposse/atmos/pkg/filesystem		coverage: 0.0% of statements
ok  	github.com/cloudposse/atmos/pkg/filetype	0.078s	coverage: 0.4% of statements in ./...
ok  	github.com/cloudposse/atmos/pkg/generate	0.685s	coverage: 7.7% of statements in ./...
ok  	github.com/cloudposse/atmos/pkg/git	0.164s	coverage: 0.3% of statements in ./...
ok  	github.com/cloudposse/atmos/pkg/github	2.462s	coverage: 0.4% of statements in ./...
ok  	github.com/cloudposse/atmos/pkg/hooks	0.264s	coverage: 7.5% of statements in ./...
ok  	github.com/cloudposse/atmos/pkg/list	2.193s	coverage: 12.0% of statements in ./...
ok  	github.com/cloudposse/atmos/pkg/list/errors	0.073s	coverage: 0.1% of statements in ./...
ok  	github.com/cloudposse/atmos/pkg/list/flags	0.072s	coverage: 0.1% of statements in ./...
ok  	github.com/cloudposse/atmos/pkg/list/format	0.119s	coverage: 0.6% of statements in ./...
ok  	github.com/cloudposse/atmos/pkg/list/utils	0.187s	coverage: 0.2% of statements in ./...
ok  	github.com/cloudposse/atmos/pkg/logger	0.161s	coverage: 0.3% of statements in ./...
ok  	github.com/cloudposse/atmos/pkg/merge	0.227s	coverage: 1.7% of statements in ./...
ok  	github.com/cloudposse/atmos/pkg/pager	0.076s	coverage: 0.9% of statements in ./...
ok  	github.com/cloudposse/atmos/pkg/perf	1.238s	coverage: 0.5% of statements in ./...
ok  	github.com/cloudposse/atmos/pkg/pro	0.177s	coverage: 0.8% of statements in ./...
ok  	github.com/cloudposse/atmos/pkg/pro/dtos	0.051s	coverage: 0.0% of statements in ./...
ok  	github.com/cloudposse/atmos/pkg/profiler	1.861s	coverage: 0.4% of statements in ./...
ok  	github.com/cloudposse/atmos/pkg/provenance	0.130s	coverage: 1.8% of statements in ./...
ok  	github.com/cloudposse/atmos/pkg/retry	0.176s	coverage: 0.2% of statements in ./...
ok  	github.com/cloudposse/atmos/pkg/schema	0.070s	coverage: 0.3% of statements in ./...
ok  	github.com/cloudposse/atmos/pkg/spacelift	0.787s	coverage: 8.4% of statements in ./...
ok  	github.com/cloudposse/atmos/pkg/stack	0.346s	coverage: 4.3% of statements in ./...
ok  	github.com/cloudposse/atmos/pkg/store	0.139s	coverage: 1.7% of statements in ./...
ok  	github.com/cloudposse/atmos/pkg/telemetry	0.518s	coverage: 2.7% of statements in ./...
	github.com/cloudposse/atmos/pkg/telemetry/mock		coverage: 0.0% of statements
ok  	github.com/cloudposse/atmos/pkg/ui/heatmap	0.129s	coverage: 0.9% of statements in ./...
ok  	github.com/cloudposse/atmos/pkg/ui/markdown	0.138s	coverage: 0.4% of statements in ./...
?   	github.com/cloudposse/atmos/pkg/ui/theme	[no test files]
ok  	github.com/cloudposse/atmos/pkg/utils	0.743s	coverage: 4.8% of statements in ./...
ok  	github.com/cloudposse/atmos/pkg/validate	1.354s	coverage: 14.5% of statements in ./...
ok  	github.com/cloudposse/atmos/pkg/validator	0.116s	coverage: 0.2% of statements in ./...
ok  	github.com/cloudposse/atmos/pkg/vender	3.308s	coverage: 3.9% of statements in ./...
ok  	github.com/cloudposse/atmos/pkg/version	0.069s	coverage: 0.0% of statements in ./...
ok  	github.com/cloudposse/atmos/pkg/xdg	0.046s	coverage: 0.1% of statements in ./...
ok  	github.com/cloudposse/atmos/tests	174.022s	coverage: 14.3% of statements in ./...
ok  	github.com/cloudposse/atmos/tests/testhelpers	90.419s	coverage: 1.1% of statements in ./...
Coverage report generated: coverage.out

references

  • Affects 9 test files with 29 cleanup handlers added
  • Modified files:
    • pkg/component/component_processor_test.go
    • pkg/describe/describe_affected_test.go
    • pkg/describe/describe_component_test.go
    • pkg/describe/describe_dependents_test.go
    • pkg/describe/describe_stacks_test.go
    • pkg/list/list_components_test.go
    • pkg/merge/merge_test.go
    • pkg/spacelift/spacelift_stack_processor_test.go
    • pkg/stack/stack_processor_test.go

πŸ€– Generated with Claude Code

Replace unconditional t.Log() calls with t.Cleanup() handlers that only
output verbose YAML/data when tests fail. This eliminates noisy stderr
output during successful test runs while preserving debug information
when needed.

Changes:
- Add 29 cleanup handlers across 9 test files
- Each handler checks t.Failed() before logging
- Include fallback to raw data (%+v) if YAML conversion is empty
- Affects component, describe, list, merge, spacelift, and stack tests

πŸ€– Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
@osterman osterman requested a review from a team as a code owner October 22, 2025 20:10
@github-actions github-actions bot added the size/m Medium size PR label Oct 22, 2025
@github-actions
Copy link

Dependency Review

βœ… No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Scanned Files

None

@osterman osterman added the no-release Do not create a new release (wait for additional code changes) label Oct 22, 2025
@codecov
Copy link

codecov bot commented Oct 22, 2025

Codecov Report

βœ… All modified and coverable lines are covered by tests.
βœ… Project coverage is 66.75%. Comparing base (a6880a8) to head (5b36099).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1704      +/-   ##
==========================================
+ Coverage   66.71%   66.75%   +0.03%     
==========================================
  Files         364      364              
  Lines       42499    42499              
==========================================
+ Hits        28354    28369      +15     
+ Misses      12056    12040      -16     
- Partials     2089     2090       +1     
Flag Coverage Ξ”
unittests 66.75% <ΓΈ> (+0.03%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 4 files with indirect coverage changes

πŸš€ New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@aknysh aknysh merged commit 7da9b3d into main Oct 22, 2025
61 of 62 checks passed
@aknysh aknysh deleted the osterman/fix-test-stderr-output branch October 22, 2025 21:22
aknysh added a commit that referenced this pull request Oct 22, 2025
Resolved conflicts in test files by adopting the new t.Cleanup pattern
from PR #1704 while maintaining fixes for golangci-lint issues.
@github-actions
Copy link

These changes were released in v1.196.0-rc.0.

osterman added a commit that referenced this pull request Oct 29, 2025
Apply PR #1704 pattern to terraform_test.go - wrap t.Log in t.Cleanup handler
that only outputs verbose terraform plan results when tests fail.

## what
- Wrapped unconditional `t.Log(output)` in t.Cleanup with t.Failed() check
- Terraform plan output now only shown when test fails

## why
- Line 230 had unconditional t.Log that dumped terraform output to CI logs
- This was pre-existing (added in May, commit d4fa2fe) before PR #1704
- Caused noisy CI logs with walls of component YAML configuration
- Follows same pattern as PR #1704 for other test files

## impact
- CI logs quieter - terraform output only shown on test failures
- Consistent with PR #1704 pattern across all test files
- Debug information preserved when actually needed

Related to PR #1704 (only log verbose test output on failure).

πŸ€– Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
osterman added a commit that referenced this pull request Oct 29, 2025
Apply PR #1704 pattern comprehensively across all test files - only show
verbose output (YAML, JSON, debug info) when tests fail.

## what
Fixed unconditional output in 6 test files:
- cmd/root_test.go: command output
- tests/cli_terraform_test.go: terraform stdout/stderr
- tests/validate_schema_test.go: schema validation output
- pkg/merge/merge_context_demo_test.go: error formatting demos
- pkg/config/command_merging_behavior_test.go: command structure debugging
- pkg/config/command_merge_core_test.go: command verification output

## why
- These tests had unconditional t.Log() or fmt.Printf() calls
- Dumped walls of YAML/JSON/debug info to CI logs on every run
- Made CI logs extremely noisy and hard to read
- Some dated back to May 2025, predating PR #1704

## how
- Wrapped all output in t.Cleanup handlers with t.Failed() checks
- Converted fmt.Printf to t.Logf for proper test integration
- Output now only appears when tests actually fail
- Preserved all debug information for troubleshooting

## impact
- CI logs dramatically quieter - no verbose output on success
- Debug info still available when tests fail
- Consistent with PR #1704 pattern across entire codebase
- Easier to spot actual problems in CI output

Related to PR #1704 (quiet test output on success).

πŸ€– Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
osterman added a commit that referenced this pull request Oct 29, 2025
Found 6 more instances of unconditional verbose logging in provenance YAML
parser tests. These tests iterate through every line of parsed YAML and log
the entire pathMap, creating walls of debug output in CI logs.

## what
Fixed debug output in 2 files:
- pkg/provenance/yaml_parser_multiline_test.go: 1 instance
- pkg/provenance/yaml_parser_arrays_test.go: 5 instances

All "=== PATH MAP ===" debug outputs now wrapped in t.Cleanup handlers.

## why
- Tests unconditionally logged every YAML line with path mapping
- Created verbose output like:
  ```
  === MULTILINE CONTEXT PATH MAP ===
  Line 0: "description: |" -> Path: "description", IsKey: true
  Line 1: "  This is" -> Path: "description", IsCont: true
  ... (dozens more lines)
  ```
- Made CI logs extremely noisy for tests that always pass
- Debug info is important for troubleshooting parser issues

## how
- Wrapped all pathMap logging loops in t.Cleanup with t.Failed() checks
- Debug output now only shown when tests actually fail
- Preserves full debugging capability when needed

## impact
- Significantly quieter CI logs for passing provenance tests
- Debug output still available when parser tests fail
- Consistent with PR #1704 pattern

Related to PR #1704 (quiet test output on success).

πŸ€– Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
aknysh added a commit that referenced this pull request Oct 30, 2025
…p handlers (#1722)

* fix: Only log terraform test output on failure

Apply PR #1704 pattern to terraform_test.go - wrap t.Log in t.Cleanup handler
that only outputs verbose terraform plan results when tests fail.

## what
- Wrapped unconditional `t.Log(output)` in t.Cleanup with t.Failed() check
- Terraform plan output now only shown when test fails

## why
- Line 230 had unconditional t.Log that dumped terraform output to CI logs
- This was pre-existing (added in May, commit d4fa2fe) before PR #1704
- Caused noisy CI logs with walls of component YAML configuration
- Follows same pattern as PR #1704 for other test files

## impact
- CI logs quieter - terraform output only shown on test failures
- Consistent with PR #1704 pattern across all test files
- Debug information preserved when actually needed

Related to PR #1704 (only log verbose test output on failure).

πŸ€– Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* fix: Wrap all unconditional test output in t.Cleanup handlers

Apply PR #1704 pattern comprehensively across all test files - only show
verbose output (YAML, JSON, debug info) when tests fail.

## what
Fixed unconditional output in 6 test files:
- cmd/root_test.go: command output
- tests/cli_terraform_test.go: terraform stdout/stderr
- tests/validate_schema_test.go: schema validation output
- pkg/merge/merge_context_demo_test.go: error formatting demos
- pkg/config/command_merging_behavior_test.go: command structure debugging
- pkg/config/command_merge_core_test.go: command verification output

## why
- These tests had unconditional t.Log() or fmt.Printf() calls
- Dumped walls of YAML/JSON/debug info to CI logs on every run
- Made CI logs extremely noisy and hard to read
- Some dated back to May 2025, predating PR #1704

## how
- Wrapped all output in t.Cleanup handlers with t.Failed() checks
- Converted fmt.Printf to t.Logf for proper test integration
- Output now only appears when tests actually fail
- Preserved all debug information for troubleshooting

## impact
- CI logs dramatically quieter - no verbose output on success
- Debug info still available when tests fail
- Consistent with PR #1704 pattern across entire codebase
- Easier to spot actual problems in CI output

Related to PR #1704 (quiet test output on success).

πŸ€– Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* fix: Wrap provenance test debug output in t.Cleanup handlers

Found 6 more instances of unconditional verbose logging in provenance YAML
parser tests. These tests iterate through every line of parsed YAML and log
the entire pathMap, creating walls of debug output in CI logs.

## what
Fixed debug output in 2 files:
- pkg/provenance/yaml_parser_multiline_test.go: 1 instance
- pkg/provenance/yaml_parser_arrays_test.go: 5 instances

All "=== PATH MAP ===" debug outputs now wrapped in t.Cleanup handlers.

## why
- Tests unconditionally logged every YAML line with path mapping
- Created verbose output like:
  ```
  === MULTILINE CONTEXT PATH MAP ===
  Line 0: "description: |" -> Path: "description", IsKey: true
  Line 1: "  This is" -> Path: "description", IsCont: true
  ... (dozens more lines)
  ```
- Made CI logs extremely noisy for tests that always pass
- Debug info is important for troubleshooting parser issues

## how
- Wrapped all pathMap logging loops in t.Cleanup with t.Failed() checks
- Debug output now only shown when tests actually fail
- Preserves full debugging capability when needed

## impact
- Significantly quieter CI logs for passing provenance tests
- Debug output still available when parser tests fail
- Consistent with PR #1704 pattern

Related to PR #1704 (quiet test output on success).

πŸ€– Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

* fix: Remove unused imports and empty loop from test refactoring

Cleaned up unused imports and code left over from wrapping fmt.Print*
calls in t.Cleanup handlers.

## what
- Removed unused `fmt` imports from 3 test files
- Removed empty `for _, cmd := range commands {}` loop

## why
Build was failing with:
- pkg/config/command_merge_core_test.go:4 - "fmt" imported and not used
- pkg/merge/merge_context_demo_test.go:5 - "fmt" imported and not used
- tests/validate_schema_test.go:5 - "fmt" imported and not used
- pkg/config/command_merging_behavior_test.go:295 - declared and not used: cmd

These were left over from converting fmt.Printf to t.Logf in t.Cleanup handlers.

## impact
- βœ… Build passes
- βœ… Tests compile successfully
- βœ… Clean code without unused artifacts

πŸ€– Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>

---------

Co-authored-by: Claude <[email protected]>
Co-authored-by: Andriy Knysh <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

no-release Do not create a new release (wait for additional code changes) size/m Medium size PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants